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SECTION  I 
INTRODUCTION 


Currently  development  programs  at  the  Naval  Training 
Equipment  Center  (NAVTRAEQUIPCEN )  have  been  concerned  with 
the  design  of  automated  training  systems  using  a  computer 
speech  recognition  technology.  This  technology  may  hold  a 
great  deal  of  promise  for  the  automated  training  of  tasks 
wherein  speech  is  the  primary  mode  of  response.  In  the 
development  of  these  systems,  computer  models  of  instructor 
behavior  have  been  created.  These  models,  similar  to  human 
instructors,  take  as  input  the  verbal  responses  of  the  stu¬ 
dent.  The  instructor  models  must  process  this  information, 
relative  to  the  ongoing  tasks  required  of  the  student,  and 
make  ational  real  time  instructional  decisions.  In  the 
instructor  model,  which  is  the  control  logic  of  the  system, 
lies  the  success  or  failure  of  the  automated  training  system 
itself. 

Recent  developments  in  basic  research  programs  show 
promise  in  enhancing  the  capabilities  of  present  instructor 
models.  This  would  include  developments  in  intelligent 
knowledge-based  training  systems  as  well  as  basic  research 
in  cognitive  processes,  and  the  representation  of  knowledge. 
The  sections  to  follow  will  review  these  recent  developments 
in  basic  research  with  the  current  designers  of  automated 
training  systems  as  an  intended  audience.  In  an  attempt 
to  facilitate  a  crossfeed  of  information  between  the  basic 
and  applied  researchers,  a  review  of  the  primary  character¬ 
istics  of  the.  speech-based  task  will  also  be  provided  with 
the  basic  researcher  as  an  intended  audience. 

In  the  first  section  to  follow,  three  exemplary  speech- 
based  tasks,  the  training  of  which  is  soon  to  be  automated, 
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are  described.  The  descriptions  are  only  brief  introductions 
designed  to  feature  the  cognitive  implications  of  training 
to  be  discussed  later.  References  are  given  for  the  reader 
desiring  more  details  on  the  tasks.  Following  the  brief 
introductions  to  the  tasks,  their  common  features  are  dis¬ 
cussed  along  with  certain  observable  differences  between 
the  novice  and  expert  which  have  implications  for  training. 

Recent  developments  in  basic  research  which  could  im¬ 
pact  the  design  of  the  instructor  model  are  reviewed  in  the 
third  section.  Technological  achievements  supported  by  the 
Office  of  Naval  Research  (ONR)  were  featured  when  relevant, 
for  programatic  continuity.  The  areas  of  basic  research 
reviewed  were  a  result  of  an  analysis  of  the  three  exemplar 
tasks.  In  that  analysis,  it  became  apparent  that  the  domi¬ 
nant  common  characteristic  of  speech-based  tasks  was  the 
time-sharing  component.  Thus  the  research  on  information 
processing  limitations  and  stages  is  reviewed  fairly  exten¬ 
sively.  Perceptual  and  motor  learning  research  is  also 
discussed  briefly  as  they  are  major  components  in  two  of 
the  speech-based  tasks.  Recent  developments  in  computer- 
assisted  instruction  (CAl)  are  also  reviewed  in  terms  of 
their  possible  implications  in  training  system  designs  for 
speech-based  tasks .  The  speech-recognition  technology 
itself  was  not  reviewed  as  our  major  concern  was  with  the 
design  of  the  instructor  model. 

In  the  fourth  section,  the  characteristics  of  an  ideal 
instructor  model  were  considered.  The  characteristics 
included  were  based  on  both  the  availability  of  specific 
technologies  identified  in  the  reviews,  as  well  as  the  needs 
identified  in  the  analysis  of  training  requirements.  The 
fifth  section  deals  with  gaps  in  technology  which  preclude 
the  immediate  development  of  our  prototype  instructor  model. 
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The  discussion  centers  around  the  practical  considerations 
of  filling  the  technological  gaps,  part  of  which  involves 
the  crossfeed  of  information  between  the  basic  and  applied 
researcher. 

The  three  exemplary  speech-based  tasks  presented 
were  selected  as  a  result  of  their  current  programs  for 
automation,  directed  by  the  Naval  Training  Eqi. ipment 
Center.  In  all  three,  the  student  is  being  trained  in 
some  form  of  air  traffic  control.  The  student  must  learn 
to  process  visual  and  auditory  information  rapidly  while 
making  verbal  advisories,  to  the  aircraft  involved.  The 
first  two  controllers  provide  for  aircraft  control  during 
landing  operations.  The  third  provides  control  during 
tactical  maneuvers . 
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SECTION  II 

CURRENT  AUTOMATED  SPEECH  BASED  TASKS 

GROUND  CONTROLLED  APPROACH  RADAR  CONTROLLER 

The  Ground  Controlled  Approach  (GCA)  term  refers  to 
both  ASR  (Aircraft  Surveillance  Radar)  and  PAR  (Precision 
Approach  Radar)  control.  The  present  context  however  »  is 
only  concerned  with  PAR  training  although  the  acronym  GCA 
is  often  used.  The  GCA  (PAR)  controllers  consist  of 
enlisted  personnel  whose  primary  responsibility  is  that  of 
monitoring  and  advising  aircraft  on  their  final  approach 
until  touchdown.  Their  task  is  to  monitor  a  radar  scope 
consisting  of  a  divided  display,  and  make  frequent  (almost 
continuous)  advisories,  according  to  rigid  vocabulary  and 
procedural  constraints. 

Figure  1  shows  an  example  of  a  PAR  display.  The  two 
lines  (cursors)  with  the  hashmarks  represent  the  glide- 
slope,  the  top  cursor  (elevation)  representing  vertical 
position,  and  the  bottom  (azimuth)  representing  lateral 
position.  The  hashmarks  themselves  represent  miles  from 
touchdown.  The  more  prominent  hashmark  is  the  five  mile 
marker.  The  aircraft  itself  is  represented  as  a  radar 
return  (target)  with  a  short  trail.  The  purpose  of  the 
trail  is  to  give  the  controller  direction  of  motion  infor¬ 
mation  as  each  radar  sweep  will  only  give  static  position 
of  the  aircraft  at  the  time  of  the  sweep.  As  can  be  seen, 
the  aircraft  (target)  is  just  inside  the  two-mile  mark, 
coming  up  on  the  glide-slope,  but  drifting  off  to  the  right. 

The  phraseology  that  would  be  used  to  advise  the  pilot 
depends  in  part  on  the  point  at  which  the  target  intersects 
the  cursor.  Figure  1  exaggerates  the  size  of  the  target 
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to  illustrate  that  it  is  broken  into  zones.  An  intersect 
in  zone  number  1  is  depicted  as  being  on  course,  while  the 
adjacent  zones  (2)  are  cues  for  the  use  of  the  adjective 
"slight"  as  in  the  statement  "slightly  right  of  course." 
Should  the  target  be  far  enough  to  the  right  of  the  c<.  sor 
that  there  is  no  intersect,  the  statement  would  be  "well 
right  of  course." 

In  addition  to  the  terminology  pertaining  to  the  azi¬ 
muth,  there  are  specific  elevation  messages  which  are  either 
statements  of  trend  or  position.  Further,  there  are  assigned 
headings  to  give  as  well  as  other  control  messages.  These 
are  summarized  in  Table  1  (Grady  and  Hicklin,  1976^). 

Position  and  trend  messages  must  be  alternated  and  phrased 
in  such  a  way  as  to  depict  movement  to  the  pilot.  Table  2 
gives  the  legal  relationships  between  trend  and  position 
messages.  Grady  and  Hicklin  (1976)1  also  cite  the  following 
additional  rules  governing  glidepath  trend  messages: 

(1)  Trend  must  be  given  between  two  different 
glidepath  position  messages . 

(2)  Trend  may  not  be  given  between  two  identical 
glidepath  position  messages. 

(3)  Two  trend  messages  must  be  separated  by  a 
glidepath  position  message. 

(4)  The  trend  message  must  be  the  correct  one 
given  the  position  message  which  follows  it 
as  shown  in  Table  2. 

The  following  rules  are  applied  to  course  correction  messages: 

(l)  Course  corrections  may  be  given  once  the  air¬ 
craft  is  on  final  approach. 
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TABLE  1 .  RADIO  TERMINOLOGY 


Glidepath  position  messages: 

«r  Approaching  glidepath 

Vfell  above  glidepath 
Above  glidepath 
Sli^itly  above  glidepath 
On  glidepath 
Slightly  below  glidepath 
Below  glidepath 
Well  below  glidepath 

Glidepath  trend  messages: 

Caning  down 
Going  below  glidepath 
Going  further  below  glidepath 
Coming  up 

Going  above  glidepath 

^  Going  further  above  glidepath 

Course  messages: 


Assigned  heading  _ 

Turn  right  heading _ 

Turn  left  heading _ 

Other  oontrol  messages: 

Begin  descent 

mile(s)  from  touchdown 
Wind  calm 
Wind  _  _  _at  _  _ 

Cleared  for  low  approach 
At  decision  height 
Execute  missed  approach 
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TABLE  2.  TABLE  OF  LEGAL  TREND  MESSAGES  SCORED  WHEN 
the  NEXT  GLIDEPATH  POSITION  IS  SPOKEN 


Legal  Trend  Messages 


Upward  Trend  Downward  Trend 


Next  Glidepath 
Position 
Message 


Going  further 

above  glidepath 

Going  further 

above  glidepath 

Going  above 
glidepath 

Coming  up 

Coming  up 

Coming  up 


Coming  down 

Coming  down 

Coming  down 

Going  below 
glidepath 

Going  further 

below  glidepath 

Going  further 

below  glidepath 


Well  above 
glidepath 

Above  glidepath 

Slightly  above 
glidepath 

On  glidepath 

Slightly  below 
glidepath 

Below  glidepath 

Well  below 
glidepath 


(2)  Outside  three  miles  from  touchdown,  the  mini¬ 
mum  course  correction  is  five  degrees  and  the 
maximum  is  ten  degrees. 

(3)  Within  three  miles  from  touchdown,  the  mini¬ 
mum  course  correction  is  two  degrees  and  the 
maximum  is  five  degrees . 

(4)  Heading  digits  and  direction  of  the  correction 
must  correspond. 

(5)  Direction  of  the  turn  must  be  correct;  i.e., 
when  the  aircraft  is  right  of  course  and  heading 
away  from  the  centerline,  a  left  turn  would  be 
correct . 
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(6)  Assigned  heading  messages  may  be  given  at  any 
time  but  the  heading  must  be  identical  to  the 
last  heading  given. 

This  short  description  of  the  GCA  controller's  duties 
is  by  no  means  exhaustive.  The  controller  also  has  other 
responsibilities  such  as  wind  and  clearance  messages,  etc. 

A  more  detailed  description  of  the  task  requirements  may 
be  found  in  the  GCA-CTS  Student  Guide  under  development 
by  Logicon  at  the  time  of  this  writing. 

As  can  be  seen  by  looking  at  the  GCA  controller's 
responsibilities,  the  phraseology  and  rules  of  production 
are  well  documented  and  standardized.  The  limited  vocabu¬ 
lary  and  definite  structure  are  an  advantage  when  consid¬ 
ering  automated  training  as  noted  by  Peuge,  Charles  and 
Miller  (197^)?  At  the  time  of  this  writing,  a  trainer 
referred  to  as  the  Ground  Controlled  Approach-Controller 
Training  System  (GCA-CTS)  has  been  produced  and  is  being 
delivered  to  the  government  for  further  test  and  evaluation. 

n 

Breaux  (1976)J  describes  the  initial  laboratory  version  of 
the  GCA-CTS  while  Barber,  Hicklln,  Meyn,  Porter  and  Slemon 
(1979)  document  the  current  version  of  GCA-CTS. 

LANDING  SIGNAL  OFFICER  (LSO) 

The  major  role  of  the  Landing  Signal  Officer  (LSO)  is 
again  that  of  aircraft  control.  In  this  case,  the  LSO  is 
not  only  an  officer,  but  also  an  experienced  pilot.  His 
duties  include  the  training  of  pilots,  and  new  LSOs  as  well 
as  the  control  of  aircraft  landings  aboard  a  carrier.  He 
is  selected  from  a  pool  of  pilots  during  either  his  advanced 
pilot  training  or  his  first  fleet  squadron  tour  of  duty. 
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His  selection  is  said  to  be  based  on: . motivation,  aviation 
ability,  and  potential  as  an  instructor.  A  detailed  des¬ 
cription  of  the  LSO's  role  can  be  found  in  a  report  by 
Hooks,  Butler,  Gullen  and  Petersen  (1978)? 

Current  training  of  LSOs  exists  in  three  phases.  The 
first  phase  consists  of  nine  working  days  of  formal  train¬ 
ing  at  Pensacola,  Florida.  It  consists  of  classroom  ses¬ 
sions  and  field  trips  to  the  extent  possible.  It  emphasizes 
the  theory  of  LSO  operations  and  information  regarding 
shipboard  equipment  and  systems  with  which  the  LSO  must 
interact.  The  second  phase  of  training  involves  skill 
development.  Here  the  trainee  gains  experience  with  obser¬ 
vation  and  control  of  Field  Carrier  Landing  Practice  (FCLP) 
operations.  The  third  phase  involves  the  observation  and 
control  of  air  wing  aircraft  aboard  ship.  The  type  of 
instruction  received  in  these  last  two  phases  is  primarily 
that  of  apprenticeship  to  senior  LSOs,  is  informal  and 
largely  unstructured.  The  length  of  time  spent  in  training 
is  hard  to  define  in  that  on-the-job  training  (OJT)  as 
defined  in  the  third  phase,  continues  on  indefinately .  The 
rate  of  training  is  dependent  on  OJT  opportunities. 

The  dependence  of  training  on  OJT  opportunities  has 
created  a  severe  man-power  shortage.  Recent  decreases 
in  carrier  deployment  (Hooks,  et  al.,  1978"*)  have  been 
cited  as  the  causal  factor  in  observed  LSO  shortages  and 
reduced  proficiency  levels  on  the  part  of  experienced  LSOs. 
This  reduction  of  carrier  landings  has  had  several  effects. 
It  has  caused  decreased  availability  for:  the  required  OJT 
needed  for  LSO  training,  the  re-establishement  of  skills 
for  the  LSO  returning  from  a  non-LSO  tour  of  duty,  the 
opportunity  to  assess LSO  performance,  and  the  enrichment 
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experiences  that  come  with  exposure  to  a  great  variety  of* 
carrier  landing  situations.  An  additional  problem  arises 
as  a  result  of  the  pilot’s  decreased  availability  to  prac¬ 
tice  with  carrier  landings.  This  reduced  proficiency  noted 
in  the  pilots'  skills  places  increased  demand  on  the  LSO’s 
performance.  For  these  reasons,  the  training  of  LSOs  has 
been  judged  a  likely  candidate  fbr  automated  training. 

Like  the  Ground  Controlled  Approach  Controller,  the 
LSO's  job  is  a  complex  cognitive  task.  Hooks',  et  al. 
(1978)^  break  his  responsibility  of  the  control  of  air¬ 
craft  landings  aboard  ship  into  four  elements:  (l)  assess 
aircraft  approach,  (2)  assess  recovery  conditions, 

^3)  direct  pilot  actions,  and  (4)  advise  superiors  of 
recovery  feasibility,  efficiency,  and  safety.  To  this  end, 
the  LSO  is  situated  on  a  platform  located  to  the  side  of 
the  ramp  and  just  aft  of  the  arresting  cables.  Figure  2 
shows  a  stylized  diagram  of  the  view  from  just  above  the 
LSO  platform.  From  this  platform,  the  LSO  must  make  visual 
judgements  of  the  aircraft's  vertical  an,d  lateral  position¬ 
ing  relative  to  range  from  carrier  and  an  optimum  glide 
slope.  To  communicate  to  the  pilot,  the  LSO  holds  a  com¬ 
munication  cradle  not  unlike  an  ordinary  telephone 
receiver  to  his  ear. 

Figure  2  gives  some  idea  as  to  the  fine  perceptual 
judgments  the  LSO  must  make,  specifically  when  the  aircraft 
is  some  distance  from  the  carrier.  After  the  final  turn, 
the  LSO  acknowledges  receiving  control  and  begins  making 
judgments  as  to  the  aircraft's  position.  The  stylized 
trail  depicting  the  trajectory  of  the  aircraft  is  to  illus¬ 
trate  gross  deviations  from  the  glidepath.  Note  that  in 
the  distance,  even  gross  deviations  by  the  pilot  would 
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require  fine  perceptual  judgments  on  the  part  of  the  LSO. 
Precise  elevation  discriminations  relative  to  an  "interna¬ 
lised  standard"  glideslope  would  be  difficult  for  the 
novice,  to  say  nothing  of  judgments  of  lateral  position. 
Example  advisories  are  shown  in  the  figure  to  illustrate 
the  type  of  vocabulary  used.  Only  samples  of  a  standardized 
vocabulary  are  shown,  but  unlike  other  types  of  controllers, 
the  LSO  may  use  nonstandard  phrases. 

The  visual  conditions  under  which  the  LSO  must  work  are 
less  than  optimal.  There  is  usually  a  haze  over  the  seas 
making  distal  objects  hard  to  see.  The  aircraft,  at  the 
beginning  of  the  glide  slope  will  appear  as  a  small  spot  in 
the  haze  with  little  apparent  movement.  The  horizon  may  or 
may  not  appear  as  well  defined  through  the  haze.  The  deck 
and  LSO  platform  will  be  in  motion  depending  on  the  sea 
conditions.  According  to  Hooks  et  al.  (1978)^  the  LSOs 
report  that  they  attend  to  changes  in  aircraft  pitch  atti¬ 
tude,  acceleration  and  engine  thrust  changes.  The  cues 
for  engine  thrust  changes  are  auditory  and  visual  (smoke). 

In  addition  to  attending  to  the  aircraft  itself,  the  LSO 
has  available  other  aids  such  as  the  Pilot  Landing  Aid 
Television  (PLAT)  for  line-up  indications  and  the  SPN-42 
radar  data  readouts  for  glide  slope,  line-up  and  speed 
indications.  Because  of  the  attention  demands  a  backup 
LSO  must  attend  to  these  other  aids  and  verbally  report 
to  the  LSO  in  charge. 

In  addition  to  the  auditory  cues  the  LSO  receives 
from  the  aircraft  itself  and  the  advisories  from  the  backup 

* 

LSO,  he  will  receive  communication  from  the  pilot  and  other 
support  personnel.  In  addition  to  his  hand-held  voice 
transmit-receive  device,  he  holds  a  device  for  activating 
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waveoff  and  cut  lights  called  a  "pickle."  Other  work  sta¬ 
tion  devices  include  the  MOVLAS  control,  a  hand  operated 
lever  for  signalling  perceived  glide  slope  positions  of 
aircraft,  and  console-located  intercom  switches  for  inter¬ 
communications  to  Air  Boss,  etc. 

There  are  several  variables  which  would  effect  the  LSO's 
performance.  The  deck  motion  (roll,  pitch,  heave,  and  yaw) 
will  effect  position  cues  relative  to  the  aircraft,  horizon 
and  plane  guard  destroyer  following  the  carrier.  Visability 
itself  may  be  attenuated  by  haze,  and  ceiling,  and  of  course 
night  landings  in  which  the  only  cues  are  the  lights  of  the 
aircraft  relative  to  the  lights  of  the  destroyer.  Voice 
communications  with  the  pilot  are  received  in  a  background 
of  other  transmissions  over  the  same  frequency  as  well  as 
verbal  calls  from  support  personnel  on  the  platform.  Addi¬ 
tionally  the  type  of  aircraft  will  vary  in  terms  of  speed 
and  maneuverability  as  well  as  varying  in  fuel  state  and 
emergency  conditions  (engine  failure,  hydraulic  system  failure, 
etc.).  Needless  to  say,  the  LSO  may  at  times  experience 
tremendous  information  load.  Hooks  et  al.  (1978):,  report 
that  the  major  determiner  of  an  LSO's  performance  rating 
is  his  ability  to  perform  under  pressure. 

AIR  INTERCEPT  CONTROLLERS  (AIC) 

Another  challenge  for  automated  speech  recognition  based 
training  systems  is  the  training  of  Air  Intercept  Controllers 
(AIC).  While  also  involved  with  air  traffic  control,  the 
AIC's  role  is  that  of  directing  an  intercept  aircraft  in  a 
combat  situation  to  destroy  an  enemy  aircraft.  Unlike  other 
air  traffic  controllers  whose  job  it  is  to  keep  aircraft  a 
safe  distance  apart,  the  AIC's  role  is  that  of  bringing  con¬ 
trolled  aircraft  together  safely  for  the  purpose  of 
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intercepting  a  target.  There  are  obviously  great  risks 
involved  when  calculating  the  interception  of  high  perform¬ 
ance  aircraft.  Multiple  aircraft  will  be  assigned  one  tar¬ 
get  to  attack  sequentially  from  different  angles.  The  AIC 
is  dealing  with  split-second  decisions  and  motor  responses 
in  the  controlling  of  these  aircraft  displayed  on  a  complex 
monitor.  His  controller  phraseology  is  more  diverse  than 
that  of  the  GCA  and  his  performance  and  learning  demands 
are  more  complicated.  The  AJC  is  generally  concerned  with 
mission-oriented  control  and  may  be  stationed  on  board  a 
ship  somewhere  near  the  mission  zone. 

The  more  recent  training  advances  have  not  yet  impacted 
the  training  of  AICs  due  to  its  complexities  although  the 
need  for  an  improved  training  situation  has  been  indicated. 
The  fleet’  <s  readiness  requirements  become  more  and  more 
difficult  to  meet  due  to  high  cost  and  low  availability  of 
live-air  training.  It  would  be  advantageous  to  have  a 
training  system  capable  of  supplementing  live-air  training 
adequately . 

AICs  are  made  up  of  officers  and  senior  enlisted  per¬ 
sonnel  trained  at  the  Fleet  Combat  Training  Centers, 
(FLECOMBATRACEN)  lo  cated  in  San  Diego.  A  class  consists 
of  students  who  work  together  for  six  w$eks  of  training. 

The  first  three  weeks  consist  of  classroom  instruction  and 
simulated  air  control.  This  training  is  supported  by  a 
training  system  developed  in  the  1 960s  referred  to  as 
Tactical  Advanced  Combat  Direction  and  Electronic  Warfare 
(TACDEW).  The  second  three  weeks  of  training  involves  the 
use  of  real  aircraft  and  pilots.  FLECOMBATRACEN  averages 
25  training  flights  per  day,  but  the  distribution  of  the 
flights  is  often  awkward  for  efficient  training.  Instructors 
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have  indicated  a  desire  for  a  training  system  capable  of 
switching  to  a  synthetically  generated  environment  during 
lag-time  between  live  intercept  missions. 

Prior  to  1977»  the  training  of  AICs  was  of  a  more  tradi¬ 
tional  method  requiring  the  mastering  of  a  technical  manual 
and  proper  button-pressing  sequence.  Now  the  course  is 
task-oriented  beginning  with  easy  tasks  and  progressing 
to  the  more  difficult.  Bach  step  in  the  job  is  explained 
and  the  equipment  is  discussed  in  terms  of  its  role  in  that 
assignment.  This  change  in  philosophy  of  training  has  been 
beneficial  to  both  the  instructor  and  the  student,  but  many 
needs  still  exist  in  the  efficient  training  of  AICs. 
FLECOMBATRACEN  and  NAVTRAEQTJIPCEN  have  proposed  the  possibility 
of  a  large-scale  training  device  to  support  AIC  training  in 
the  1980s  which  is  directed  at  alleviating  many  of  the 
training  problems  (Grady,  Hicklin,  and  Miller,  1977^). 

A  sketch  of  a  possible  training  device  similar  to  that 

ry 

proposed  by  Halley,  King,  and  Regelson  (1979)'  is  shown  in 

\ 

Figure  3»  Consoles  differ  between  ships  so  this  one  i3  not 
necessarily  representative  of  all  the  different  types  of 
consoles.  It  will  serve  to  demonstrate  an  apparatus  with 
which  an  AIC  must  interact  to  achieve  the  required  intercept. 
Centered  on  the  console  is  the  display  of  all  raw  radar 
return.  Superimposed  on  this  is  a  display  of  a  limited 
amount  of  computer  graphics  or  symbology.  The  numerous 
control  panels  around  the  radar  display  regulate  various 
functions  such  as  the  expansion  or  off-set  of  the  radar  dis¬ 
play,  the  building  of  symbology  on  the  display,  the  locking 
on  of  the  symbols  on  the  radar  display,  etc*  A  track  ball 
controls  the  position  of  a  tab  (similar  to  a  cursor  on  a  CRT) 
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on  the  display.  A  head  set  is  worn  which  is  open  to  all 
transmissions  on  a  given  frequency  and  a  foot  control  is 
used  for  keying  the  mike.  A  number  entry  pad  is  used  for 
the  entry  of  data  and  there  are  numeric  displays  to  be 
monitored. 

Before  assuming  control  of  the  monitor,  the  AIC  attends 
a  briefing  which  will  inform  him  of  the  nature  of  the  on¬ 
going  or  anticipated  mission(s)  or  training,  maneuvers,  and 
various  other  pertinent  information  such  as  weather  condi¬ 
tions,  topographical  problems  such  as  moyintains,  etc.  He 
spends  some  time  then  actually  viewing  the  radar  return,, 
listening  to  transmissions  and  generally  getting  an  overall 
impression  of  what  kinds  of  return  on  the  display  are  static 
(mountains)  and  what  are  dynamic  (aircraft).  Upon  assuming 
the  responsibility  of  the  controls,  the  AIC  must  identify 
aircraft  entering  his  scope  by  means  of  a  check-in  proce¬ 
dure  with  the  aircraft.  The  Combat  Air  Patrol  (CAP)  possess 
a  transponder  which  continuously  squawks  a  code.  The  AIC 
"challenges"  the  new  return  on  his  screen  through  a  series 
of  motor  responses  and  identifies  the  code  the  CAP  is 
squawking  by  a  display  on  the  lower  left-hand  section  of 
the  console.  He  then  procedes  to  build  a  CAP  symbol  and 
then  "hooks"  the  symbol  so  that  the  computer  will  continue 
to  make  calculations  for  that  assigned  symbol.  By  building 
symbology  in  this  manner,  he  is  allowing  the  computer  to 
keep  track  of  the  interceptor  in  a  rather  complex  display. 
This  is  followed  by  the  entry  of  three  multi-digit  codes 
on  the  number  entry  pad,  and  the  depressing  of  the  order- 
send  button.  The  check-in  is  then  completed  with  a  radio 
check  by  the  AIC . 

When  a  "bogey"  (unidentified  aircraft)  is  detected, 
the  AIC  communicates  with  his  assigned  aircraft  to  vector 
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him  to  the  nearest  point  of  intercept.  He  must  then  con¬ 
tinue  to  provide  the  pilot  updated  information  as  to  the 
bogey’s  bearing,  range,  track  and  ground  speed.  Sudden 
changes  in  the  bogey's  parameters  require  new  vectors  to 
be  calculated  and  then  reported  to  the  pilot.  While  plot¬ 
ting  and  controlling  the  intercept,  he  must  be  constantly 
aware  of  the  range,  bearing,  track  and  ground  speed  of 
other  aircraft  in  the  vicinity  of  the  controlled  aircraft. 
He  must  also  be  verbally  responding  to  communications 
from  the  pilot(s). 

Although  this  is  obviously  an  abbreviated  synopsis 
of  the  AICs  multidimensional  tasks,  we  can  nonetheless 
gain  some  insight  and  appreciation  for  the  complexity  of 
the  training  situation.  An  expert  AXC  related  in  a  per¬ 
sonal  communication  some  of  the  difficulties  he  encountered 
in  training.  In  a  general  sense,  mastering  the  automated 
system  and  all  the  manual  inputs  required  to  accomplish 
multiple  intercepts  coupled  with  the  mental  and  verbal 
demands  was  very  difficult  until  some  state  of  automaticity 
was  reached.  In  a  little  more  detailed  sense,  he  identi¬ 
fied  six  difficult  areas  of  mastery:  (l)  properly  setting 
up  the  scope  initially,  (2)  properly  identifying  all  the 
"players , "  ( 'j)  punching  in  the  correct  responses  or  buttons 
of  his  scenario  to  accomplish  the  mission  without  taking 
his  eyes  off  the  scope,  (4)  the  pacing  demands  relative  to 
the  speed  of  the  target  (usually  very  fast,  e.g.  reconnais¬ 
sance  aircraft  traveling  over  2000  mph),  (5)  understanding 
the  computer  programs  in  the  system  and  how  to  use  them  to 
accomplish  the  goal,  and  (6)  the  phraseology. 
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COMMON  FEATURES  OF  THE  THREE  CONTROLLER  TASKS 

The  three  controller  roles  just  outlined,  represent  the 
beginning  point  For  automated-speech-recognition-based  train¬ 
ing  technology.  All  three  of  these  job  roles  are  currently 
experiencing  a  man-power  shortage  in  the  Navy,  and  could 
benefit  from  automated  training.  Present  training  methods 
vary  for  the  three  jobs  but  all  begin  with  a  school  designed 
to  introduce  the  students  to  their  subsequent  duties.  As 
Nowell  (1978)  points  out,  there  is  limited  training  time 
associated  with  these  schools.  There  may  be  an  instructor 
shortage  as  in  the  case  of  training  AXCs,  and  the  instruc¬ 
tional  cost  per  student  is  quite  high.  Once  the  student 
leaves  the  school,  OJT  opportunities  for  further  training 
may  be  on  the  decline  as  reported  by  Hooks  et  al.  (1978)"* 
in  the  case  of  LSOs . 

All  three  training  programs  share  a  common  mode  of 
response — that  of  speech.  The  speech  requirement  may  serve 
as  a  task  dimension  that  will  set  these  roles  apart  from 
other  military  roles.  As  one  might  surmise,  if  the  student's 
response  mode  is  verbal  (a  mode  which  is  quite  slow),  other 
resources  could  simultaneously  be  required.  A  point  which 
we  will  attempt  to  make,  will  be  that  these  tasks  require 
the  student  to  perform  various  processing  functions  (and 
possibly  motor  responses)  concurrently  with  speech  produc¬ 
tion  thus  comprising  a  multi-task  situation. 

Now  a  second  characteristic  of  these  tasks,  not  unre¬ 
lated  to  the  multi-task  feature,  is  that  a  major  portion  of 
the  tasks  performed  by  controllers  are  event-driven,  meaning 
that  the  pace  of  the  task  is  controlled  by  external  events. 

It  follows  from  the  nature  of  multi-task  situations  that 
the  pace  of  events  may  at  times  challenge  the  student's 
resource  capacities.  Thus  the  objective  of  an  instructor 
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model  in  these  situations,  would  be  to  not  only  build  know¬ 
ledge  structures  within  the  student  using  CAI  technology, 
but  also  to  develop  certain  processing  skills  via  adaptive 
training  technology  to  allow  for  effective  resource  utiliza¬ 
tion. 


The  primary  stimulus  mode  for  all  three  tasks  is  visual 
with  intermittent  verbal  inputs  from  pilots  and  possibly 
support  personnel.  The  visual  input  may  be  of  the  complexity 
that  it  requires  the  training  of  perceptual  and  visual  search 
skills.  All  visual  cues  are  above  threshold  with  possibly 
some  minor  exceptions.  As  in  the  case  of  visual  displays, 
the  complexity  may  foster  divided  as  well  as  focused  atten¬ 
tion  deficits.  More  will  be  said  about  this’  later  as  the 
controller’s  tasks  are  analyzed  in  more  detail. 

The  ability  to  process  information  from  a  complex 
display,  select  and  execute  a  verbal  response  in  seemingly 
parallel  functions,  seems  to  be  at  the  heart  of  the  con¬ 
troller's  skills.  Hooks  et  al.  (1978)^  report  that  the 
major  determiner  of  an  LSO ' s  abilities  is  his  ability  to 
perform  under  pressure  without  getting  "flustered"  or 
"confused."  Stated  in  other  ways,  this  becomes  his  ability 
to  process  and  respond  accurately  under  heavy  information 
load.  The  ability  to  process  efficiently  large  amounts  of 
visual  (and/or  auditory)  inputs  concurrently  with  speech 
production  at  a  rapid  pace,  is  the  major  difference  between 
the  novice,  and  the  highly  skilled  controllers.  These  are 
the  characteristics  of  the  controller's  tasks  which  an 
automated-speech  based  instructor  model  must  address. 

The  core  of  an  automated  training  system  is  the  instruc¬ 
tor  model,  by  which  we  mean  the  control  logic  in  the  system 
which  in  many  ways  may  emulate  a  human  instructor.  In  using 


J 


NAVTRAEQUIPCEN  79-C-0085-1 


the  human  instructor  as  a  point  of*  departure,  we  would  note 
that  the  instructor  is  usually  an  expert  in  the  field  in 
which  he  instructs.  Having  been  a  student  once  himself, 
he  may  have  insights  as  to  what  he  was  like  as  a  novice, 
what  he  is  like  as  an  expert,  and  some  opinions  as  to  the 
process  which  transformed  him  from  a  novice  to  an  expert 
state.  He  can  then  use  these  insights  to  guide  his  instruc¬ 
tional  decisions  which  he  makes  during  the  instructional 
process.  These  insights  and  past  experiences  are  incorpor¬ 
ated  into  his  internal  logic  as  it  were.  He  in  effect, 
understands  the  process  through  which  the  student  is  going. 

Before  creating  an  instructor  model,  we  must  understand: 
the  expert,  the  novice,  and  the  technical  difference  between 
the  two,  in  a  way  that  will  allow  us  to  deduce  the  control 
logic  for  an  automated  trainer.  Since  the  developers  of 
training  systems  are  usually  not  subject  matter  experts  (SME), 
the  development  process  is  usually  preceded  by  a  period  of 
preparatory  investigation,  sometimes  called  task  analysis. 
Usually  a  task  analysis  involves  collaboration  between  SMEs 
and  the  technical  staff  responsible  for  system  development. 
Often  this  is  merely  a  breakdown  of  the  task  into  subtasks, 
duties  and  skills  which  the  student  is  to  acquire  by  interact¬ 
ing  with  the  system.  This  level  of  analysis  may  not  imply 
specific  training  techniques  to  be  used. 

In  the  sections  to  follow,  we  will  present  a  task 
analysis  based  on  current  basic  research  issues  in  process 
and  structure  which  is  to  lead  to  the  beginning  of  an 
instructor  model  for  automated  speech  recognition  based 
training  systems.  It  is  to  be  kept  in  mind  that  the  purpose 
of  the  analysis  model  is  provocative  in  nature.  Ideas  and 
positions  are  proposed  which  may  remain  unvalidated  until 
empirical  support  is  found.  We  will  propose  that  the  skills 
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and  abilities  of  the  expert  ought  to  be  analyzed  in  some 
detail,  and  contrasted  with  an  analysis  of  the  novice's 
inabilities,  errors,  omissions,  etc.  These  contrasts  should 
be  done  in  such  a  way  as  to  suggest  certain  training  techni¬ 
ques,  and  instructional  strategies.  For  purposes  of  the 
present  paper,  the  analyses  of  the  controller  tasks  were 
carried  out  using  the  applied  technical  reports  cited, 
interviews  with  students  and  instructors,  and  the  basic 
research  literature  (concentrating  where  applicable  on 
the  efforts  supported  by  ONR) . 


PRELIMINARY  OBSERVATIONS  OF  THE  EXPERT  AND  NOVICE 

The  present  authors  have  followed  the  development  of 
GCA-CTS  somewhat  since  its  inception,  and  have  interviewed 
the  chief  warrant  officer  monitoring  the  development.  This 
individual  was  not  only  an  "expert"  controller,  but  was  also 
the  person  in  charge  of  PAR  instructors.  In  trying  to  des¬ 
cribe  his  behavior  during  an  approach,  he  found  introspection 
difficult  as  so  much  of  it,  in  his  words  "is  automatic,"  He 
added  however,  that  the  students  do  not  reach  this  "automatic" 
level  in  the  short  two  weeks  of  PAR  training  in  which  the 
training  system  is  to  be  utilized.  In  this  two  week  training 
period,  the  students  would  receive  only  12-15  hours  of  actual 
practice  on  the  display.  Following  the  formal  training 
period,  they  would  receive  another  1 60  hours  or  so  of  OJT. 

In  describing  the  characteristics  of  his  proficiency, 
the  expert  stated  that  he  "does  alot  of  looking  ahead."  His 
experience  allows  him  to  anticipate  what  the  next  event  will 
be.  Upon  interrogating  the  expert  further,  he  described  the 
"looking  ahead"  as  an  anticipation  of  a  response  rather  than 
a  perceptual  anticipation.  With  his  experience  and  the 
information  he  "carries  in  his  head"  (such  ass  the  type  of 
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plane,  wind  direction  and  velocity,  thermals,  etc.)  he  Is 
able  to  anticipate  what  the  target  will  do  next  on  the  dis¬ 
play  and  what  his  probable  next  call  will  be. 

The  PAR  students  present  quite  a  different  picture  than 
the  expert.  We  had  only  limited  opportunity  to  observe  two 
students  at  the  beginning  of  their  training  on  the  scope. 

At  this  time  the  display  was  simplified  wi'fti the  removal  of 
the  elevation  cursor  leaving  only  the  azimuth  to  which  the 
student  was  to  respond  (see  Figure  l).  Even  with  the  simpli¬ 
fied  display,  the  most  common  errors  with  the  beginning  stu¬ 
dent  were  caused  by  his  inability  to  process  the  information 
rapidly  enough. 

Listed  below  are  examples  of  eight  types  of  errors 
which  were  observed.  For  the  first  example,  it  was  found 
that  students  could  not  compute  and  give  course  headings 
fast  enough.  If  the  verbal  responses  were  to  be: 

TURN  RIGHT  HEADING.  .ONE.  .FIVE .  .FIVE . SIX.. MILES  FROM.. 

the  student  would  respond: 

TURN  RIGHT  HEADING.  .ONE.  ..  .FIVE . FIVE . SIX 

Since  the  level  of  speech-recognition  technology  used 
at  the  time  did  not  have  connected-speech  capabilities,  the 
students  were  required  to  make  short  pauses  between  phrase . 

V 

segments  and  numbers.  The  beginning  student  however,  was 
observed  pausing  between  numbers,  presumably  performing  mental 
calculations,  for  a  much  longer  duration  them  required  by  the 
technology.  These  "concentration  pauses"  had  several  effects. 
By  the  time  the  last  digit  was  given  the  pilot  was  late  in 
executing  his  turn.  This  meant  that  additional  corrections 
were  then  required  of  the  student  to  bring  the  pilot  back 
on  course . 
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A  resulting  error  was  that  the  pauses  would  cause  him 
to  miss  the  next  call  such  as  giving  the  pilot  his  range  to 
touchdown.  The  students  stated  that  they  could  tell  they 
were  missing  the  range  call,  but  "the  next  call  was  required 
to  get  the  pilot  on  course,  and  there  simply  wasn't  enough 
time."  Thus  the  first  two  types  of  errors  are  related,  those 
being  inevitable  pauses  and  the  resulting  missed  calls. 

A  third  type  of  error  was  that  the  headings  given  were 
sometimes  inaccurate.  When  the  students  were  asked  to  intro¬ 
spect  on  their  thought  processes  in  computing  the  headings, 
they  responded  that  they  tried  to  visualize  the  aircraft  (as 
it  intersected  a  line  to  the  runway)  as  if  they  were  looking 
down  on  it.  Knowing  the  aircraft's  present  heading  (an 
estimate),  and  the  runway  heading  (e.g.  150°  and  160° 
respectively),  they  then  used  this  visualization  to  determine 
that  a  right  turn  was  needed  with  a  ten  degree  correction. 
Further,  they  know  that  at  this  point  in  the  approach  the 
corrections  should  be  given  in  five  degree  units.  Finally 
they  would  conclude  that  the  first  of  the  corrections  should 
be  a  right  turn  to  155°»  The  process  they  describe  would 
be  assumed  to  be  quite  time  consuming  whereas  the  expert 
states  that  he  is  not  really  aware  of  any  thought  processes, 
he  just  does  it~rapidly. 

A  fourth  characteristic  of  the  novice  may  be  referred 
to  as  excessively  long  dwell  times .  He  will  fixate  on  some 
portion  of  the  display  (presumably  while  he  is  slowly  pro¬ 
cessing  the  information  in  order  to  make  a  response)  and 
neglect  other  portions  of  the  display.  As  an  example,  when 
both  the  elevation  and  the  azimuth  are  present  on  the  display 
(see  Figure  l)  the  student  may  dwell  on  elevation  call(s) 
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and  neglect  the  scanning  of  the  azimuth.  The  result  of 
course  is  that  the  pilot  may  slowly  drift  undetected  off 
course. 

A  fifth  characteristic  is  that  the  student  will  give  a 
correct  call  but  in  the  wrong  form.  It  will  be  recalled 
that  the  PAR  controller's  calls  must  be  made  within  rather 
strict  verbal  guidelines  (see  Tables  1  and  2).  Thus  the 
student  may  transmit  the  incorrect  phrase,  "How  do  you 
read  me?"  when  the  correct  phraseology  should  have  been, 

"How  do  you  hear  me?" 

The  students  report  an  additional  phenomenon  we  will 
term  the  sixth  characteristic.  They  report  that  they  will 
lose  certain  information  that  they  are  supposed  to  "carry 
in  their  heads"  (presumably  short-term  memory)  for  subse¬ 
quent  use.  For  example,  just  before  the  handoff,  (when  ^ 

the  student  takes  control  of  the  aircraft  from  another 
controller)  the  aircraft's  current  heading  is  transmitted. 

When  he  finally  takes  control  and  is  required  to  compute 
a  course  correction,  he  discovers  to  his  dismay  that  he 
can't  remember  the  aircraft's  current  heading  from  which 
he  must  compute  the  change. 

A  seventh  observation  was  that  when  processing  load 
increases,  the  student '9  voice  quality  degrades.  The  quality 
is  of  course  important  for  controller  communication,  but  even 
more  troublesome  for  the  automated-speech-based  training 
system  is  the  fact  that  changes  in  voice  quality  preclude 
the  system  from  identifying  the  transmission.  If,  for  example, 
the  student  is  getting  behind  in  his  calls,  he  becomes  con¬ 
fused  as  to  how  he  should  handle  the  situation.  He  may  begin 
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to  speak  softly,  change  intonation  and  pause  in  awkward 
places  in  the  phrases.  His  hesitancy  may  be  manifested 
by  his  inadvertently  mumbling  "Uhmm..."  in  place  of  desig¬ 
nated  pauses.  This  creates  an  input  such  as  "GOING  UHMMFUR- 
THER. . .ABOVEUHMMGLIDEPATHUHH . . . . "  which  the  system  cannot 

recognize. 

A  final  error  is  described  by  the  student  as  "not 
knowing  what  to  do."  He  will  make  a  wrong  call  and  then  dis¬ 
play  surprise  that  it  was  wrong  or  he  will  panic  during  an 
approach,  turn  to  his  instructor,  and  ask, "What  am  I  sup¬ 
posed  to  do  in  this  case?"  An  example  situation  is  when 
an  aircraft  appeared  on  the  display  on  a  heading  quite 
discrepant  (e.g.  210°)  from  the  approach  heading  of  160°. 

The  surprised  student  attempted  to  issue  a  course  correction 
of  five  degrees  followed  by  a  second  course  correction  of 
five  degrees  and  so  on.  In  the  middle  of  his  second  ad¬ 
visory  the  student  realized  that  the  target  would  intersect 
and  surpass  the  cursor  by  the  time  he  could  issue  enough 
course  corrections  in  five  degree  units.  He  began  to  hesi¬ 
tate  in  his  speech  and  stammer  somewhat  as  he  helplessly 
watched  the  aircraft  fly  off  the  display.  It  became 
apparent  after  discussing  it  with  his  instructor  that  he 
simply  did  not  have  the  knowledge  about  how  to  handle  that 
situation. 

These  eight  types  of  errors,  though  not  exhaustive 
serve  to  illustrate  the  initial  problems  of  the  novice.  The 
last  type  of  error  we  would  like  to  refer  to  as  a  struc ture 
problem.  By  structure  we  mean  in  short,  "knowledge,"  al¬ 
though  the  concept  of  structure  will  be  developed  more 
fully  in  the  sections  to  follow.  In  this  case,  errors  are 
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committed,  or  maybe  a  response  was  omitted,  because  of  a 
lack  of  "knowledge"  (structure)  on  the  student's  part. 
Predictably,  if  thb  student  had  read  and  reviewed  his  manual 
before  getting  on  the  trainer,  we  found  few  errors  of  this 
sort.  Only  if  the  manual  had  not  been  reviewed  did  the 
student  make  many  errors  from  the  standpoint  of  a  lack  of 
knowledge. 

Most  of  the  errors  we  observed,  we  will  refer  to  as 
processing  errors.  We  propose  that  the  first  seven  types 
of  errors  committed,  were  caused  by  the  slow  rate  of  pro¬ 
cessing  on  the  part  of  the  novice.  The  students  could  tell 
us  after  an  approach,  the  responses  that  should  have  been 
made,  so  we  assume  they  possessed  the  "knowledge."  They 
would  comment  that  "there  simply  wasn't  enough  time"  or 
"too  many  things  were  happening  at  once."  Thus  in  the 
present  analysis,  we  would  propose  that  the  first  seven 
characteristics  of  the  novice  are  caused  by  processing 
limitations.  The  longer- than- required  pauses  in  speech  were 
caused  by  the  student  not  being  able  to  speak  and  think 
concurrently.  The  student's  slow  rate  of  processing  causes 
hist  to  fall  behind  in  his  event-driven  task  which  in  turn 
causes  him  to  miss  some  of  the  calls.  Two  other  types  of 
errors,  those  of  incorrect  phraseology  and  incorrect  heading 
computation,  we  will  argue  are  often  the  result  of  incom¬ 
plete  processing.  That  is,  because  of  the  paced  nature  of 
their  task,  the  students  feel  that  they  must  respond  though 
they  feel  "hurried"  and  have  not  had  enough  time  to  fully 
process  the  response.  It  was  felt  that  the  excessively 
long  dwell  times  were  similar  in  nature  to  the  pauses  in 
speech— that  of  not  being  able  to  attend  to  additional 
elements  of  the  display  while  processing.  The  loss  of 
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information  from  short-term  storage  may  also  be  due  to 
excessive  processing  load  inhibiting  the  rehearsal  or 
the  recycling  of  the  information.  Finally  the  degradation 
in  speech  quality  could  logically  be  due  to  the  student's 
inability  to  process  and  speak  simultaneously  when  the 
load  becomes  heavy.  Thus  in  observing  the  novice,  and 
interviewing  the  expert,  we  concluded  that  any  instructor 
model  for  aircraft  controller  training  systems  would  need 
to  be  concerned  with  the  effect  of  training  on  processing 
skills  in  addition  to  the  building  of  knowledge  structures. 
The  inclusion  of  process  considerations  makes  the  automated 
speech-based  systems  unique  from  the  more  traditional  CAT 
approaches . 
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SECTION  III 

RECENT  DEVELOPMENTS  IN  BASIC  RESEARCH 

The  usage  of  verbal  reports  from  both  the  expert  and 
the  novice  can  yield  valuable  insights  on  training  methods 
and  instructional  logic  for  incorporation  into  a  subsequent 
instructor  model.  Reliance  on  verbal  reports,  however,  may 
be  quite  limited.  Neither  the  expert  controller,  nor  the 
novice  are  familiar  with  the  terminology  found  in  a  cognitive 
science.  One  gets  a  definite  feeling  when  questioning  the 
students  that  the  questions  are  often  leading,  and  the 
responses  often  reported  in  a  form  that  fits  our  own  internal 
schemata.  In  our  thinking,  the  validity  of  the  verbal  re¬ 
ports  should  be  treated  as  tenuous  and  only  used  as  a  point 
of  departure  for  further  verification.  This  subsequent 
verification  would  quite  naturally  follow  from  any  empirical 
data  we  might  be  able  to  find.  In  the  sections  to  follow, 
we  examine  the  basic  research  literature  to  see  what  it  has 
to  offer  in  the  development  of  an  instructor  model.  We  will 
start  by  choosing  to  make  a  distinction  between  process  and 
structure  as  their  requisite  training  technology  would  differ. 

PROCESSING  LIMITATIONS 

A  major  difference  between  the  expert  and  the  novice  is 
that  the  expert  seems  to  be  able  to  simultaneously  handle 
multiple  inputs,  hold  information  in  memory,  process  the 
information,  and  select  the  next  response  concurrent  with  the 
execution  of  a  verbal  response.  The  novice,  on  the  other 
hand,  seems  to  be  analogous  to  the  person  who  can't  walk  and 
chew  gum  at  the  same  time.  The  novice  seems  to  be  held  back 
by  a  limited  central  processing  capacity.  The  concept  of  a 
limited  capacity  can  be  traced  back  to  the  attention 
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literature  of  the  1950’s  and  1960*s  with  the  single-channel 
model  of  Broadbent  (1958)^  and  the  filter  position  of 
Treisman  (1960)^  as  exemplars.  These  early  models  put  the 
source  of  the  proverbial  bottleneck  on  the  attention  stage 
of  the  process.  Subsequently,  Deutsch  and  Deutsch  (1963)^ 
proposed  that  the  bottleneck  occured  later  during  the 
decision  or  response  stages.  Shiffrin  (1976)  continued 
this  notion  referring  to  the  concept  as  capacity  limitations, 
while  others  refer  to  it  as  resource  limitations. 

Kahneman  (1973)  J  proposed  a  resource  competition 
model  that  assumed  that  all  cognitive  tasks  draw  from  the 
same  pool  of  limited  resources.  Thus,  if  a  particular  task 
demands  a  large  share  of  the  resources,  there  will  be  little 
left  over  if  a  second  task  is  required.  This  reasoning  led 
some  researchers  (e.g.  Posner  and  Boies,  1971^)  to  use  a 
reaction  time  (RT)  probe  as  a  measure  of  the  processing 
demands  of  a  particular  task.  The  idea  was  that  as  the 
resource  demands  of  a  primary  task  increased,  the  longer  the 
RT  to  the  secondary  probe  task.  The  benefit  of  this  approach 
could  be  that  different  primary  tasks  (or  the  same  primary 
task  at  different  stages  of  learning)  could  be  assessed 
in  terms  of  their  relative  processing  demands  by  their 
probe  RT  measures.  Some  reservation  in  this  methodology  for 
assessing  the  demands  of  primary  tasks  lies  in  the  findings 
that  visual  and  auditory  probes  lead  to  different  patterns 
of  results  (Schwartz,  19761'*)  while  different  responses  to 
the  probes  (manual  vs.  vocal)  also  lead  to  different 
results  (McCleod,  1978^). 

A  second  methodology  of  interest,  termed  the  "secondary 
task  paradigm"  was  developed  and  is  discussed  by  Kerr  ( 1973)1 
Here  the  subject  is  given  two  concurrent  tasks.  One  is 
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designated  the  primary  task  and  the  subject  is  told  that  a 
certain  level  of*  performance  on  this  task  must  be  maintained. 
The  other  task,  deemed  the  secondary  task,  is  allowed  to 
fluctuate  in  performance  level.  These  two  tasks  are  naturally 
assumed  to  compete  for  the  same  central  resources.  As  the 
demands  of  the  primary  task  are  increased,  the  performance 
of  the  secondary  task  declines  while  the  primary  task's 
performance  remains  invariant .  Thus  the  secondary  task  is 
used  as  a  measure  of  the  processing  demands  of  the  primary 
level. 


In  the  past,  two  positions  have  been  taken  in  these 
dual  task  situations.  One  view  (Kahneman,  1973  is  that 
these  (vague)  limited  resources  may  be  viewed  as  a  pool  of 
"effort"  which  may  be  allocated  among  tasks.  This  common 
undifferentiated  pool  of  "effort”  is  drawn  upon  from  essen¬ 
tially  all  tasks,  no  matter  how  diverse.  The  second  view 

1  8 

(Posner  and  Keele,  1970  )  is  that  some  tasks  compete  for 

1 7 

the  same  resources  while  others  do  not.  Kerr  (1973) 
concluded  that  two  very  different  categories  of  what  was 
referred  to  as  mental  operations  may  not  require  the  same 
central  mechanism  at  all  and  may  be  executed  in  parallel 
(such  as  encoding  and  executing  a  movement  to  a  physical 
stop) .  This  parenthetical  example  would  of  course  be 
pertinent  to  a  segment  of  the  AIC  task,  that  of  visual 
encoding  of  the  display  cues,  during  the  execution  of  but¬ 
ton  pressing  responses.  Other  of  Kerr's  categories,  she 
concedes,  do  compete  and  interfere  with  each  other. 

More  recently,  ONR  has  funded  some  projects  which 
looked  at  the  resource  limitation  problem.  Hunt,  Lansman, 
and  Wright  (1979)^  report  additional  support  for  the 
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general  notion  of*  Kahneman  (1973)  J  in  that  relatively 
diverse  tasks  showed  performance  decrement  when  combined. 

The  Hunt,  et.al.  (1979)^  report  is  also  of  interest  in 
that  "reasoning”  was  one  of  the  mental  operations,  whereas 
most  of  the  previous  work  utilized  perceptual  or  motor 
reaction  time,  or  short-term  memory  paradigms.  In  one 
experiment,  Hunt,  et.al.  (1979)^  used  non-verbal  intelli¬ 
gence  test  problems  of  varying  difficulty  level  paired  with 
a  simple  psychomotor  task.  They  found  the  performance  on 
the  secondary  psychomotor  task  to  decline  just  prior  to 
the  first  error  on  a  test  problem  as  the  problems  got 
progressively  more  difficult.  This,  of  course,  would  have 
implications  in  adaptive  training  in  that  the  secondary 
task  could  be  used  as  a  difficulty  level  index  when  error 
measurement  on  the  primary  task  proves  insensitive.  These 

paradigms  have  also  been  used  to  assess  individual  differences 

20 

in  processing  capacity  (see  Lansman,  1978  ). 

The  possibility  of  using  secondary  tasks  as  indirect 

performance  measurement  sounds  tempting,  but  there  are 

1 7 

further  considerations.  As  Kerr  (1973)  1  and  Posner  and 
/  \  1 4 

Boies  (1971)  point  out,  sometimes  task  A  interferes  with 
task  B  but  not  vice-versa,  while  sometimes  two  tasks  don't 
interfere  with  each  other  at  all.  These  findings  are  further 
used  for  the  contention  that  at  least  one  of  these  tasks  in 
these  situations  do  not  require  the  same  central  processing 
capacity.  "If  they  did,  they  both  would  be  expected  to 
interfere."  (Posner  and  Boies,  1971 »  P»  407  ^) •  As  a 
response  to  this,  Norman  and  Bobrow  (1975)  developed 
their  concepts  of  data-limited  and  resource-limited  pro¬ 
cesses.  In  their  view,  whenever  the  performance  on  a  task 
can  be  increased  with  a  increased  allocation  of  resources 
(e.g.  processing  effort,  or  concentration),  then  the  task 
was  said  to  be  resource-limited.  Whenever  the  performance 
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level  remains  invariant  to  increased  allocations  of  processing 
effort  (resources),  the  task  is  said  to  be  data— limited. 

Here  the  performance  is  limited  by  either  poor  signal 
quality  (e.g.  low  signal-to-noise  ratio)  which  was  referred 
to  as  Signal  data-limits  or  an  inadequately  stored  memory 
representation  referred  to  as  Memory  data-limits. 

21 

Norman  and  Bobrow  describe  what  they  referred  to  as 
a  performance-resource  function  in  which  task  performance 
is  related  to  a  single  hypothetical  dimension  of  resource 
allocation.  Figure  4  shows  resource  allocation  functions 
for  two  tasks.  As  additional  resources  are  allocated 
moving  from  point  A  to  point  B  on  the  abscissa,  performance 
increases  on  both  tasks  and  are  resource-limited  at  all 
points  in  that  range.  But  as  the  allocation  is  further 
increased  from  points  B  to  C,  no  further  increase  in  task  I 
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ia  produced;  thus  task  X  is  now  data-limited  while  task  IX 
is  still  resource-limited.  At  point  C,  task  II  makes  the 
transition  to  data-limitation.  Beyond  point  C,  further 
allocations  of  resources  will  not  effect  performance,  only 
changes  in  the  tasks  themselves  (e.g.  increasing  signal-to- 
noise  ratio)  will  produce  performance  changes. 

The  performance-resource  function  shown  in  Figure  k 
shows  performance  on  both  tasks  increasing  as  increases  in 
resources  are  allocated  to  both  simultaneously.  But  the 
underlying  assumption  is  that  the  resources  are  common  to 
both  tasks  and  limited.  Thus  the  only  way  that  we  can 
increase  allocation  of  resources  to  task  I  is  to  take  some 
of  the  resources  from  task  II  and  vice-versa.  Thus  in  the 
figure,  as  the  horizontal  coordinate  on  the  task  I  function 
moves  to  the  right,  the  horizontal  coordinate  on  the  task  II 
function  would  be  required  to  move  to  the  left.  In  the 
region  from  A  to  B,  movements  of  the  horizontal  would  also 
imply  movements  on  the  vertical.  As  can  be  seen,  increases 
in  performance  on  one  task  due  to  increases  in  resource 
allocation  would  cause  decreases  in  resources  and  possibly 
decreases  in  performance  on  the  other  task.  The  actual 
relation  to  performance  would  depend  on  the  region  of  the 
abscissa  included  in  the  investigation. 

Resource  allocation  functions  are  largely  hypothetical 

21 

since,  as  Norman  and  Bobrow  point  out,  the  problem  would 
be  in  the  control  and  measurement  of  the  allocation  dimension. 
But  the  performance  of  one  task  can  and  has  been  empirically 
related  to  the  performance  of  another  task.  The  left  panel 
of  Figure  5  shows  such  a  function  between  two  tasks  and  is 
referred  to  as  a  Performance  Operating  Characteristic  (POC), 
The  corresponding  resource  allocation  function  in  the 
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Resource  Allocation 


Figure  5  .  Resource  Allocation  Functions  and  the  Corresponding  POC 


right-most  panel  shows  that  the  region  of  investigation  on 
the  abscissa  is  limited  to  the  A  to  B  region,  referring 
back  to  Figure  4.  As  can  be  seen,  as  resources  are  taken 
from  one  task  and  applied  to  the  other,  the  performance  on 
the  first  task  drops  while  the  performance  on  the  second 
increases.  This  would  be  a  typical  POC  wherein  both  tasks 
are  resource  limited. 

If  the  investigation  were  expanded  to  include  the 
abscissa  region  shown  in  Figure  4  to  be  A  to  C,  then  the 
resulting  POC  would  take  a  different  form  as  shown  in 
Figure  6.  As  can  be  seen,  in  the  region  from  B  to  C  on  the 
abscissa,  taking  resources  from  task  X  to  give  to  task  II 
has  no  effect  on  the  performance  of  task  I*  Taking  from 
task  II  to  give  to  task  I,  however,  does  effect  task  II* s 
performance.  Thus  if  the  range  of  our  investigation  were 
restricted  to  the  region  between  B  and  C,  we  might  conclude 
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ranges  between  C  and  D  it  can  be  seen  from  the  right-hand 
panel  that  the  two  tasks  would  appear  to  be  independent  of 
each  other.  Increasing  the  resource  allocations  on  task  II 
from  C  to  D  while  decreasing  the  allocations  on  task  I  from 
D  to  C  effects  the  performance  on  neither  task.  Thus  if  the 
difficulty  level  on  task  II  were  to  increase ,  requiring  an 
increased  effort  on  the  controller's  part  in  order  to  main¬ 
tain  a  constant  performance  level,  the  performance  on  task  I 
would  not  be  effected,  leading  to  the  conclusion  that  the 
two  tasks  do  not  interfere  with  each  other  nor  do  they  com¬ 
pete  for  the  same  central  mechanism.  Norman  and  Bobrow's 
(1975,  1976  )  point  is  that  tasks  may  have  been  investi¬ 

gated  in  a  region  wherein  one  or  the  other  or  both  are  data- 
limited  and  not  resource-limited.  Thus  the  position  espoused 

•I  O 

by  Kahneman  (1973)  J  that  there  exists  a  very  general  pool 
of  limited  cognitive  resources  from  which  all  tasks  must 
draw,  is  quite  plausible— despite  the  findings  that  some 
tasks  appear  to  be  independent  of  each  other  and  processed 
in  parallel. 

One  may  ask — How  then  may  we  account  for  the  fact  that 
the  expert  controller  seems  quite  facile  with  concurrent 
inputs  and  responses?  The  expert  describes  his  responses 
as  being  quite  automatic  in  nature,  requiring  little  or  no 
conscious  processing  effort.  Norman  and  Bobrow  explain 
that  with  practice,  the  students  may  learn  to  become  more 
efficient  in  their  processing,  maybe  by  eliminating 
processing  steps  or  learning  only  to  process  the  minimum 
relevant  data,  etc.  Thus  with  learning,  the  student 
approaches  his  data-limited  asymptote  more  rapidly  as  he 
increases  his  resource  allocations.  This  is  depicted  in 
Figure  8.  With  a  resource  function  as  is  shown  in  function 
£,  the  expert  performing  this  task  would  seem  to  be  able  to 
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Figure  8.  Changes  in  the  Resource  Allocation 
Functions  in  Various  Stages  of 
Training  A  Through  D 

do  so  independently  of  other  tasks  in  that  with  very  little 
processing  effort,  he  achieves  data-limitation  status.  Thus 
as  described  here,  the  performance  on  this  task  may  not 
really  be  "automatic”  but  simply  "cheap"  in  terms  of  the 
amount  of  resources  required  to  reach  asymptote. 

21  22 

An  extension  of  Norman  and  Bobrov’s  (1975»  1976  ) 

theory  of  resource-limited  processes  with  implications  for 

training,  comes  from  a  group  of  efforts  funded  by  the 

United.  States  Air  Force  Office  of  Scientific  Research  (AFOSR) 

23 

(see  Gopher,  Williges,  Williges,  and  Damos,  1975,  Gopher 
and  North,  1977?^  Wickens  and  Gopher,  1977?^  Lintern  and 
Gopher,  1978?^  Gopher  and  Navon,  1978?^  Navon  and  Gopher, 
1978?**  Navon  and  Gopher,  1979^)*  Navon  and  Gopher  (1979)^ 
present  a  theory  of  resource  allocation  based  on  concepts 
from  microeconomics.  Without  reviewing  it  in  great  detail 
at  this  point,  they  present  several  concepts  which  may  prove 
helpful  in  establishing  insights  into  instructional 
strategies . 
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The  first  is  a  concept  which  they  refer  to  as  Subiect- 
Task  Parameters  which  is  a  characterization  of  a  subject 
relative  to  a  specific  task  at  a  given  point  in  time.  These 
parameters  would  be  an  aggregation  of  indices  representing 
predictability  of  the  stimulus  events,  response  complexity, 
amount  of  practice,  stimulus-response  compatibility,  signal- 
to-noise  ratio,  etc.  The  set  of  parameters  would  subsume 
Norman  and  Bobrow's  specifications  of  signal  data-limi ts . 
memory  data-limits  and  more.  Performance  on  the  specified 
task  then  is  said  to  be  a  joint  function  of  all  the 
subject-task  parameters  and  the  momentary  amount  of  resources 
allocated  to  the  task. 

A  second  concept  of  interest  is  the  task's  Demand  for 
resources.  Not  to  be  confused  with  task  difficulty,  demand 
is  defined  as  the  amount  of  resources  required  (given 
constant  subject-task  parameters)  to  achieve  a  specified 
performance  level.  Task  Difficulty,  on  the  other  hand,  is 
defined  as  resource  efficiency  and  given  by  the  average 
slope  of  the  performance-resource  function.  Assuming  that 
the  tasks  represented  back  in  Figure  4  can  be  compared  on 
the  same  performance  scale,  it  would  follow  that  task  IX 
is  more  difficult  than  task  I.  For  a  fixed-unit  increment 
in  resources  allocated,  at  a  particular  point  on  the 
abscissa,  task  I  yields  a  greater  marginal  increase  in 
performance  than  task  II.  Damand,  however,  would  be  the 
minimum  amount  of  resources  required  (e.g.  the  amount 
represented  by  the  distance  from  points  A  to  B)  to 
attain  a  predetermined  level  of  performance  (e.g.  the 
asymptote  for  task  i) . 

29 

Finally  Navon  and  Gopher  introduce  the  concept  of 
an  Indifference  curve.  Indifference  curves  (or  equal 
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utility  contours)  are  concepts  borrowed  from  economics. 

When  the  composite  demand  of  two  conjoint  tasks  exceed  the 
resources  available,  then  the  student  must  decide  how  to 
allocate  the  insufficient  resources  among  the  two  tasks  in 
such  a  way  as  to  maximize  his  subjective  utility  relative 
to  the  joint  performance.  Figure  9  shows  the  POC  of 
Figure  5  with  a  set  of  hypothetical  indifference  curves 
superimposed.  Points  a  and  b  represent  two  specific  points 
on  the  indifference  function  u^  and  on  the  POC.  The  point 
a  represents  high  performance  on  task  II  relative  to  task  I 
presumably  because  the  student  has  invested  more  of  his 
resources  in  task  II  than  task  I,  whereas  the  inverse  is 
true  for  point  b.  Because  both  points  a  and  b  reside  on 
the  same  indifference  curve  however,  we  would  say  that  both 
points  represent  the  same  level  of  subjective  utility 
(satisfaction  with  the  joint  performance  level)  to  the 
student,  and  that  he  is  indifferent  about  which  combination 


u,  u,  u3 
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of  individual  task  performance  levels  to  choose.  In 
fact,  all  points  (including  c.)  on  u^  represent  the  same 
utility  level.  All  points  below  u^ ,  say  point  ja,  represent 
a  lower  utility  level  which  the  student  would  attempt  to 
raise  by  investing  additional  resources.  The  points  above 
Uj  represent  higher  utility  levels  which  it  is  assumed  the 
student  would  seek.  However,  the  POC  itself  represents  the 
upper  limit  to  the  combination  of  resource  investments  the 
student  can  make.  Thus,  it  would  follow  that  the  student 
would  select  the  combination  of  resource  investments 
yielding  the  joint  performance  associated  with  the  highest 
possible  utility  level.  This  would  of  course  be  point  d 
on  u^.  The  performance  levels  displayed  at  this  point  in 
time  would  be  Px  and  Py  as  shown. 

The  importance  of  the  indifference  curves  for  the 
development  of  training  systems  is  that  they  can  be  manipu¬ 
lated  by  the  instructor.  Priorities  on  the  subtask  perfor¬ 
mance  levels  can  be  altered  via  a  payoff  matrix  or  by  simple 
instructions.  This  is  essentially  what  is  being  done  in 
the  dual- task  studies  reviewed  by  Kerr  (1973)  and  Hunt, 
et.al.  (1979)^  when  one  task  is  designated  as  primary  on 
which  the  performance  level  is  to  remain  invariant,  and  a 
second  task  on  which  performance  is  allowed  to  fluctuate. 

In  this  case,  the  indifference  curves  would  represent  a 
complete  lack  of  trade-off  as  shown  in  the  top  panel  of 
Figure  10.  By  increasing  the  difficulty  level  of  task  I, 
the  POC  is  changed  as  is  the  point  in  which  it  intersects 
the  indifference  curve.  But  since  there  is  a  complete 
lack  of  trade-off  in  utility  between  different  performance 
levels  of  the  two  tasks,  and  all  u^  overlap  at  the  minimum 
performance  level  Px,  the  only  adjustment  the  student  can 
make  is  to  take  resources  from  task  II  to  invest  in  task  I 
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Figure  10.  Indifference  Curves  with  Indifferent 
Tradeoff  Properties 


in  order  to  maximize  instructed  utility  in  which  Px  is  con¬ 
stant.  With  the  drain  of  resources  invested  in  the  secondary 
as  the  primary  task  increases  in  difficulty,  the  secondary 
performance  levels  decline  from  Py  to  Py" . 

If  the  instructions  to  the  student  to  simply  consider 
task  I  as  primary  are  not  completely  successful  in  establish¬ 
ing  indifference  curves  with  complete  lack  of  trade-off 
properties,  they  might  look  similar  to  the  ones  shown  in 

^7 
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the  lower  panel  in  Figure  10.  In  this  case,  as  the  diffi¬ 
culty  level  in  task  1  is  increased,  and  the  student  seeks 
to  redistribute  his  resources  to  attain  maximum  subjective 
utility,  performance  levels  in  both  tasks  are  affected.  It 
is  notewortly  though,  that  the  performance  on  the  secondary 
task  (Py  to  Py”)  may  still  be  more  sensitive  to  variations 
in  the  primary  task  difficulty  levels  than  the  performance 
on  the  primary  task  itself  (Px  to  PXM). 

This  is  one  possible  explanation  for  the  phenomena 
reported  by  Hunt  et.  al  (1979)^  wherein  they  commented 
that  frequently  dual-task  studies  are  unable  to  hold  the 
performance  on  the  primary  task  constant.  There  are  of 
course  other  explanations  which  we  cannot  explore  here, 
but  there  is  one  other  which  is  worth  considering.  Thus 
far  we  have  assumed  that  the  student  is  able  to  fully 
control  his  resource  allocations.  If  this  assumption  were 
wrong  (e.g.  the  secondary  task  is  distracting  enough  that 
it  commands  a  minimum  amount  of  attentional  resources),  the 
student  may  not  be  able  to  shift  enough  of  his  resources 
to  the  primary  task  to  keep  Px  invariant  even  if  those  were 
his  direct  instructions  and  he  is  given  explicit  feedback 
on  his  primary  task  performance. 

It  is  worth  noting  at  this  point  that  the  treatment  of 
the  human  processing  system  as  the  simple  resource  alloca¬ 
tion  theories  just  described,  makes  no  mention  of  process¬ 
ing  stages  and  makes  only  weak  assumptions  about  the  nature 
of  the  resources  themselves.  The  assumptions  that  are 
required  however,  are  given  in  Navon  and  Gopher  (1979)  ~ 
which  the  reader  is  encouraged  to  consult.  Because  of  the 
generality  of  our  consideration  of  resources  and  tasks,  the 
conclusions  to  be  drawn  should  prove  to  be  quite  robust  with 
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respect  to  a  vide  variety  of  training  situations.  This 
would,  of  course,  be  of  benefit  to  those  developing  training 
systems,  as  the  time  constraints  may  be  tight  enough  to  pre¬ 
clude  a  thorough  analysis  of  each  specific  task. 

RESOURCE  MANIPULATION  IN  TRAINING 

Consider  for  a  moment  the  situation  wherein  the  student 
is  confronted  with  a  dual-task  situation  which  he  must 
master.  We  of  course  have  not  considered  the  case  of  three 
of  more  conjoint  tasks  because  of  the  complexity.  So  con¬ 
sider  the  two  tasks  possibly  as  complex  composite  tasks 
into  which  simpler  tasks  may  be  mapped.  As  the  student 
begins  training,  we  might  consider  the  source  of  his  task 
priorities  and  his  resultant  indifference  functions.  One 
source  would  be  the  student’s  own  biases,  shaped  by  his 
previous  experience— perhaps  as  a  pilot.  A  second  source 
would  be  the  terminal  task  requirements  set  by  the  job 
specifications.  As  an  example,  in  PAR  training,  the  stu¬ 
dents  are  told  that  the  target’s  position  relative  to 
elevation  is  more  important  than  the  aximuth.  In  fa'  c, 
the  students  are  given  importance  ratios  which  would  indi¬ 
cate  such.  A  third  such  source  could  be  momentary  priori¬ 
ties  set  dynamically  by  the  training  system  for  pedogogical 
purposes.  Human  instructors  are  often  observed  telling  the 
students  such  things  as,  ’’This  time  concentrate  more  on 
task  A  than  task  B.”  This  can  be  done  in  extreme  when  the 
subject  is  told  to  concentrate  on  one  part  of  the  display 
while  ignoring  the  other.  In  fact,  the  display  may  be 
modified  so  that  the  part  of  the  display  to  be  ignored  is 
blank.  This  extreme  position,  where  the  tasks  are  learned 
in  isolation  rather  than  conjointly,  brings  up  the  old  part- 
whole  training  problem  (  for  older  reviews  see  Adams,  I960?0 
Fleishman,  1965?^and  Freedle,  Zavala  and  Fleishman,  1968^). 
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The  assumption  implied  in  making  the  student  concen¬ 
trate  on  one  task  at  the  expense  of  others  is  that  learning 
rate  is  positively  related  to  the  amount  of  resources  in¬ 
vested.  In  some  tasks  this  assumption  would  be  a  foregone 
conclusion  in  that  the  human  organism  is  no  longer  viewed 
as  a  passive  learner.  In  information  storage  problems,  the 
learner  must  invest  resources  into  such  activities  as 
elaborative  processing  as  opposed  to  mere  maintenance 

O  O 

rehearsal  (see  Craik  and  Lockhart,  1972JJ).  This  assump¬ 
tion  would  then  suggest  the  canoe pt  of  a  Resource  Allocation- 
Acquisition  Function  as  shown  in  Figure  11.  As  can  be  seen, 
the  momentary  learning  rate  is  portrayed  (in  this  case  arbi¬ 
trarily)  as  a  function  of  the  amount  of  resources  allocated. 

In  this  view,  the  rate  at  which  a  task  is  mastered  could  be 
altered  by  manipulating  the  resources  allocated.  The  resources 
could  in  turn  be  influenced  by  the  task  priorities  given  the 
student  by  the  instructor. 


Figure  11.  Exemplar  Resource  ALlocation-Aoquistion  Function 
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There  are  at  least  three  exemplar  studies  in  which 

task  priorities  in  a  dual-task  paradigm  were  manipulated 

(Gopher  and  North,  197ka^  Gopher  and  North,  197*lh?'*  and 

2k  \ 

Gopher  and  North,  1977  )•  These  studies  were  unique 

relative  to  the  rest  of  the  dual— task  literature  in  that 
they  manipulated  task  priorities  during  the  student’s 
training  on  the  tasks  rather  than  asymptotic  dual-task 
performance.  Gopher  and  North  (1977  )  paired  a  one- 

I 

dimensional  tracking  task  with  a  digit-processing, 
reaction-time  task,  under  the  following  priority  conditions: 
(1.0,0),  (.7, .3),  (.5, .5),  (.3, .7),  (0,1.0).  The  (1.0,0) 
and  (.0,1.0)  comparisons  were  of  course  single-task 
presentations  while  the  others  were  dual-task  presentations. 

It  was  found  that  tracking  performance  displayed  larger 
improvements  with  the  larger  priorities  (1.0,  and  .7)  than 
with  the  more  nominal  levels  of  (.5)  and  (.3)*  The  digit 
processing  task  however  was  relatively  insensitive  to 
priority  manipulations .  In  their  analysis,  improvements  in 
the  digit-processing  task  were  found  only  in  the  time¬ 
sharing  conditions  and  were  attributed  to  an  improvement  in 
general  time-sharing  skills.  Thus  for  them,  the  optimal 
priorities  for  training  dual-task  performance  was  (.7) 
for  tracking  and  (.3)  for  digit  processing. 

Consider  the  hypothetical  resource  allocation- 
acquisition  functions  for  the  two  tasks  as  plotted  against 
each  other  as  shown  in  Figure  12.  As  diagrammed,  once  the 
student  invests  some  minimal  level  of  resources  in  the  digit¬ 
processing  task,  the  marginal  return  for  each  additional  unit 
of  resources  diminishes .  Not  so  for  the  tracking  task,  however, 
which  shows  the  momentary  acquisition  rate  to  be  highly  sensi¬ 
tive  to  resource  investment  in  the  region  between  ( .5)  and 
(.8).  If  the  human  instructor,  with  his  training  experience 
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was  aware,  even  roughly,  of  these  functions  he  would  instruct 
the  student  to  concentrate  primarily  on  his  tracking,  while 
processing  the  digits  when  he  can. 

An  intelligent  automated  training  system  could  also 

manipulate  the  student's  resources  in  real  time.  A  set  of 

techniques  for  making  decisions  similar  to  these  was  sug- 

36 

gested  by  Chant  and  Atkinson  (1973)  and  is  discussed  by 
Chatfield  and  Gidcumb  (1977)?^  Borrowing  from  some  of  the 
concepts  espoused  by  Chant  and  Atkinson  (1973)?**  suppose 
that  the  terminal  priorities  (post-training,  in- the- field 
priorities)  are  (.5,  »5).  Suppose  also  that  the  control 
logic  of  the  automated  training  system  is  allowed  to  set 
the  priorities  throughout  training  and  that  the  student  is 
given  adequate  feedback  regarding  the  joint  performance 
levels  (see  Gopher  and  North,  1977  about  the  feedback 
problem).  Then  the  system  could  seek  a  division  of 
priorities  (analogous  to  Chant  and  Atkinson's  turnpike 
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solution”)  during,  training  which  would  optimize  the  average 
acquisition  rate.  The  term  "turnpike  solution”  was  borrowed 
from  economics  and  was  itself  gin  analogy  in  that  the  most 
rapid  trajectory  from  one  point  to  the  next  may  not  be  the 
shortest  (straight-line)  path.  The  quickest  way  may  be  to 
detour  to  the  turnpike,  take  the  turnpike  at  a  more  rapid 
rate,  then  exit  the  turnpike  taking  another  path  to  your 
final  destination.  Thus  in  our  dual— task  case,  the  system’s 
solution  might  be  to  train  the  student  first  on  the  tracking 
task  in  isolation  in  order  to  attain  some  minimum  level  of 
proficiency  (the  path  to  the  turnpike),  then  put  the  student 
in  the  dual-task  situation  with  the  (.7*  .3)  priorities 
(the  rapid  turnpike),  followed  by  a  final  short  period  of 
training  with  equal  priorities  (path  from  the  turnpike) 
until  the  joint  performance  criterion  is  attained.  This 
round-about  solution  should  be  quicker  than  simply  training 
the  student  under  equal  priorities  from  the  beginning. 

Again  it  should  be  noted  that  present  treatment  of  the 
human  processing  system  is  in  very  general  terms  with  only 
weak  assumptions  about  the  specific  nature  of  the  tasks. 

We  only  assume  that  in  the  dual-task  situation  there  is 
some  unspecified  amount  of  competition  for  processing 
resources  between  two  tasks.  The  relationship  between  the 
two  tasks  may  vary  from  one  extreme,  that  being  complete 
task  independence  wherein  there  is  no  extra  cost  for  concur¬ 
rence,  to  finally  the  case  of  task  incompatibility  wherein 
an  extra  cost  for  concurrence  is  found.  We  will  propose  in 
later  sections  that  an  intelligent  system,  using  the  general 
principles  discussed,  be  designed  to  dynamically  converge  on 
an  optimal  solution  regarding  certain  instructional  strate- 

24 

gies.  In  the  present  example  of  the  Gopher  and  North  (1977) 
study,  the  system  should  be  able  to  handle  the  training  of 
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two  quite  divergent  tasks,  one  being  a  continuous  event* 
driven  tracking  task  requiring  an  adaptive  logic  (using  the 
ratio  of  acceleration  rate  determinants  of  the  control 
dynamics  as  an  adaptive  variable  to  keep  RMS  error  within 
tolerance),  with  the  other  being  a  discrete  self-paced  digit¬ 
processing  task.  The  intelligent  system, shpuld  be  able  to 
direct  the  student's  resources  even  to  the  extent  of  giving 
one  of  the  tasks  as  priority  of  zero  by  not  introducing  it 
until  a  later  time.  This  control  process  could  be  extended 
to  the  decision  processes  of  knowing  when  to  add  a  third 
task  as  the  marginal  momentary  acquisition  rates  of  the 
first  two  tasks  falls  below  a  particular  level.  A  more  com¬ 
plete  discussion  of  this  control  logic  will  be  presented 
later.  There  are  of  course  some  assumptions  such  as: 
the  ability  of  the  subject  to  direct  his  resources,  the 
acquisition  of  resource  management  skills,  and  the  nature 
of  the  performance  feedback  requisite  for  resource  manage¬ 
ment.  For  these  reasons  we  deem  it  necessary  to  examine 
the  components  of  the  various  tasks  in  which  controllers 
may  be  required  to  engage,  the  nature  of  their  demand  for 
resources,  and  their  probable  costs  for  concurrence. 


PROCESSING  STAGES 

Recently  there  has  been  a  resurgence  of  interests  in 
"stage"  theoretical  approaches  to  the  information  processing 
problem.  Basically  the  proposition  is  that  processing  takes 
place  in  discrete  and  independent  functional  stages.  Further, 
these  stages  are  generally  considered  to  be  successive  in 
nature.  Thus  as  Sternberg  (1969/  espoused,  the  reaction 
time  (RT)  of  a  response  may  be  decomposed  and  attributed  to 
a  set  of  individual  subprocesses.  The  underlying  assumption 
is  that  only  one  component  process  may  be  active. at  any  one 
time.  This  we  will  refer  to  as  the  Discrete  Stage  Model. 

It  is  noteworthy  that  a  recent  alternative  offered  by 
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McClelland  (1979)J~  assumes  that  all  components  of  a 
processing  system  operate  continuously,  but  pass  information 
from  one  process  to  the  next  as  it  becomes  available.  This 
he  refers  to  as  the  Cascade  Model. 

Until  nov  our  analysis  has  not  obligated  us  to  break 
the  general  pool  of  resources  into  processing  components. 

We  could  continue  in  that  vein  and  still  find  it  useful,  in 
fact  helpful,  in  deriving  general  instructional  strategies 
with  robust  qualities  relative  to  a  wide  variety  of  tasks. 

In  fact  we  hope  to  show  that  the  basic  research  efforts 
investigating  the  separate  stages,  have  led  to  similar 
conclusions.  The  first  two  stages  discussed  are  visual  en¬ 
coding  and  search  and  comparison.  These  are  followed  by  a 
discussion  of  internal  representation  and  its  effect  on 
the  processing  of  the  inputs  and  the  selection  of  a  response 
Finally,  response  execution  is  discussed  focusing  on  motor 
responses  because  of  its  relevance  to  tasks  such  as  those 
required  of  the  AIC. 

ENCODING  OR  PERCEPTUAL  STAGE.  The  terms  "coding"  and 
"encoding"  have  been  used  in  various  ways  throughout  the 
literature.  In  our  usage,  we  refer  to  the  process  by  which 
the  raw  stimulus  data  is  transformed  (or, coded)  into  a  form 
representative  of  an  item  in  memory.  As  an  example,  the 
LSO  viewing  the  approaching  aircraft  in  the  distance,  as 
is  shown  in  Figure  13,  encodes  the  visual  data  in  a  way 
which  matches  a  representation  leading  to  further  processing 
The  distant  features  may  include  shape  of  the  aircraft's 
silhouette,  the  distance  from  the  horizon,  apparent  movement 
etc.  In  processing  these  cues,  e.g.  shape  of  the  silhouette 
a  code  is  formed  which  matches  an  entity  in  memory.  The 
representation  in  memory  may  indirectly,  through  further 
processing,  lead  to  the  execution  of  a  particular  verbal 
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Figure  1 3*  Exemplar  Silhouettes  of  Two  Approaching 

Aircraft:  One  vith  a  High  Pitch  Angle  and 
the  Other  *a.th  a  Low  Pitch  Angle 


advisory  on  the  part  of  the  LSO.  It  may  be  that  the  silhouette 
was  that  of  an  aircraft  with  the  pitch  angle  too  low.  What¬ 
ever  the  form  of  its  corresponding  code,  it  matches  a 
representation  in  memory  which  eventually,  with  further 
processing,  leads  to  the  advisory  "attitude." 

The  beginning  student  may  not  be  able  to  perceive  the 
subtle  cues  of  a  distal  silhouette  as  an  aircraft  needing 
to  keep  his  "nose  up."  In  fact  the  student  would  be  pre¬ 
cluded  from  perceiving  it  no  matter  how  much  of  his  re¬ 
sources  he  invested  in  the  perceptual  process.  Thus  we 
would  say  the  process  is  signal-data  limited.  What  is  re¬ 
quired  at  this  point  is  Perceptual  Learning.  LaBerge 
(1976)  0  describes  a  perceptual  learning  model  with  three 
stages  (these  are  learning  stages  and  should  not  be  con¬ 
fused  with  processing  stages).  The  first  stage  he  des¬ 
cribes  as  Feature  Discovery  wherein  the  student  becomes 
sensitive  to  particular  attributes  of  the  sensory  patterns. 
Gibson  (1969)  referred  to  this  process  as  differentiation. 

The  ability  is  acquired  by  noting  differences  in  a  variety 
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of  sensory  patterns.  Two  silhouettes  are  shown  in 
Figure  13  which  depict  an  approaching  aircraft.  Most 
observers  using  the  similarities  can  readily  identify 
the  objects  as  aircraft ,  but  only  with  practice  can  one 
begin  to  estimate  the  pitch  angles  from  the  silhouettes. 

In  fact  some  effort  is  required  to  enumerate  the  differences 
in  the  patterns  such  as  the  aircraft  with  the  greater  pitch 
angle:  having  thicker  wings,  showing  less  tail,  being  a 
taller  figure,  etc. 

If  the  features  the  LSO  uses  were  identified  then  it 
is  obvious  that  a  part  of  the  training  task  would  be  to 
point  out  the  relevant  features.  The  problems  in  identi¬ 
fying  the  features  are  reported  by  Hooks,  Butler,  Gullen, 
and  Petersen  (1978)5  The  verbal  reports  from  the  expert 
LSOs  (the  main  source  for  the  developers  of  training 
systems)  are  only  partially  helpful.  Wofk  needs  to  be 
carried  out  on  methodological  approaches  for  identifying 
relevant  features.  Some  research  to  thip  end  is  being 

carried  out  by  Gilson  and  Owen  at  the  Ohio  State  Univer- 

/  *42 

sity  Aviation  Psychology  Laboratory  (1979).  When  the 
features  are  not  known  however,  the  old  time-worn  method 
of  mere  practice  with  feedback  should  work.  It  would  be 
expected  however  that  the  resource  allocation-acquisition 
function  would  be  a  non-uniform  distribution,  i.e.  the  * 
feature  discovery  process  on  the  part  of  the  student 
would  require  resources,  and  that  the  skill's  acquisition 
rate  would  depend  on  the  resources  invested  by  the  student 
as  directed  by  the  system. 

40 

A  second  stage  of  LaBerge's  perceptual  learning 
model  is  referred  to  as  coding.  Here  a  subset  of  the 

features  are  dealt  with  as  a  group  and  assigned  a  code. 

3 

In  Neisser's  (1976)  J  view,  this  process  is  schema-driven, 
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i.e.  the  student  has  developed  sort  of  an  outline  or  list  of 
features  for  which  he  explores  the  optic  array.  As  long  as 
the  perceptual  schemata  supports  fine  distinctions  between 
patterns  such  as  those  in  Figure  13,  they  are  kept.  If  the 
schemata  fail,  however,  they  are  modified. 


A  third  stage  assumed  by  LaBerge,  which  is  of 
considerable  interest  for  instructors,  is  that  of 
Automatic  Coding.  Much  of  LaBerge 's  efforts  (LaBerge, 

19731  LaBerge  and  Samuels,  197^"*  LaBerge,  1976^°)  are 
invested  in  this  concept  of  automaticity .  In  short,  with 
extensive  practice  the  combination  of  separate  features 
are  unitized  or  treated  as  a  single  perceptual  unit.  In 
other  words,  the  expert  LSO  would  automatically  code  the 
left-hand  silhouette  as  a  complete  unit  in  some  way  that 
would  represent  "the  nose  being  down"  without  the  time  and 
resource  consuming  exploration  of  the  separate  features. 

The  importance  of  this  unitizing  process  and  the  concept  of 
automaticity  is  that  it  implies  very  fast  reaction  times, 
and  little  or  no  demands  for  any  central  resources.  If 
no  demands  at  all  were  required  in  automatic  coding,  the 
process  would  presumably  be  carried  out  in  parallel  with 
other  activities.  More  will  be  said  about  the  concept  of 
automaticity  in  the  next  section. 

At  this  point,  it  might  be  worth  noting  the  role  of 

expectations  in  this  perceptual  process.  It  will  be 

recalled  that  the  expert  controller  reports  that  wit:h  his 

experience  he  can  to  some  extent  predict  the  events  before 

they  occur.  Thus  he  is  thinking  ahead,  in  an  attempt  to 

anticipate.  It  is  hypothesized  that  this  "anticipation  or 

expectation"  would  facilitate  the  perceptual  process. 

/  \kk 

LaBerge  (1973)  assumes  a  parallel  hierarchical  model  in 
which  the  sensory  input  is  first  analyzed  by  feature 
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detectors.  The  results  of  the  analysis  by  feature  detectors 
Is  then  organized  Into  codes  (e.g.  letters).  The  codes  are 
further  organized  Into  higher  level  codes  (e.g.  words,  res¬ 
ponses,  etc.)  Now  it  is  assumed  that  not  only  may  the 
hierarchy  be  activated  from  the  bottom-up  via  the  sensory 
input,  but  may  also  be  activated  top-down  via  the  memory 
system.  Phrased  in  other  terminology,  if  the  hierarchies 
are  viewed  as  schemata,  we  would  say  that  the  schema  may 

Jj  Q 

be  concept-driven  as  well  as  data  driven  (Neisser,  197o  _>). 

Specifically,  the  advantage  of  anticipation  is  that 

the  hierarchical  network  is  transformed  into  a  heightened 

state  of  excitability.  The  LSO  would  expect  to  see  the 

nose  drop  on  an  aircraft  which  had  just  been  given  an 

advisory  that  he  was  above  glideslope.  Thus  when  a  certain 

visual  pattern  of  cues  is  expected,  the  features  are  coded 

rapidly  and  sent  to  the  memory  system  for  a  match  or  perhaps 

in  the  case  of  a  highly  practiced  hierarchy,  directly  to  a 

response  for  execution.  The  power  of  anticipation  is  demon- 

/  v  44 

strated  by  LaBerge  (1973)  wherein  the  response  latencies 
in  a  letter-matching  task  were  found  to  be  the  same  between 
novel  n letter-like"  symbols  and  familiar  (and  highly  prac¬ 
ticed)  actual  letters,  when  the  specific  letters  or  symbols 
were  expected.  If  the  letters  or  symbols  were  unexpected 
however,  only  the  ordinary  letters  were  processed  rapidly 
as  if  they  were  automatic  and  unitized.  It  would  be  of 
interest  to  know  for  training  purposes,  whether  the  expert 
Lso's  rapid  processing  of  visual  data  is  due  more  to  automa- 
tieity  or  expectation.  This  could  presumably  be  explored 
by  taking  RT  measures  on  expected  and  unexpected  events. 

Not  surpri singly,  the  instructional  logic  would  depend  on 
the  degree  to  which  automat i city  was  the  goal  relative  to 
expectation. 
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SEARCH  AND  COMPARISON.  Sternberg  ^966)^  devised  a  memory 
search  paradigm  which  has  now  become  a  classic.  In  this 
paradigm,  as  the  experimenter  presents  a  series  of  characters 
to  the  subject,  one  at  a  time,  the  subject  is  to  check  each 
one  and  indicate  whether  or  not  the  character  is  a  member 
of  a  set  for  which  the  subject  was  instructed  to  watch. 

The  set  of  items,  for  which  the  subject  was  to  check,  may 
be  referred  to  as  the  memory  set  with  the  elements,  (the 
objects  of  the  search)  the  targets .  The  memory  set  of  items 
was  usually  given  to  the  subject  just  before  a  longer  series 
of  items  (termed  the  input  items)  was  presented.  The  list 
of  input  items  contained  items  which  were  not  in  the  memory 
set,  and  were  termed  the  distractor  set  with  elements  being 
called  dlstractors .  With  this  paradigm,  Sternberg  found 
that  the  RT  to  each  input  item  was  a  function  of  the  number 
of  items  in  the  memory  set.  As  an  example,  a  set  of  input 
items  which  might  be  presented  to  the  subject  one  at  a 
time  would  betafkbaghzgabw.  .  .  The  set  of 
items  for  which  he  is  searching  (the  memory  set)  is 
(a,  b,  h)  and  each  of  the  members  is  called  a  target.  In 
the  list  of  input  items  we  have  underlined  the  targets. 

The  rest  of  the  letters  would  be  the  dlstractors. 

The  theory  emanating  from  these  findings  was  that  the 
subject  performed  a  memory  search  with  each  input  item,  i.e. 
when  an  item  was  presented,  the  subject  would  compare  it  in 
a  serial  fashion  with  each  member  of  the  memory  set.  Over 
the  years,  there  arose  some  controversy  (which  we  won't 
review  here)  as  to  whether  the  subject  terminated  the  search 
as  soon  as  he  found  a  match,  or  whether  he  continued  the  com¬ 
parison  process  until  the  memory  set  was  exhausted.  Of 
interest  in  the  present  context  was  that  the  reaction  time 
required  for  the  subject  to  perform  this  memory  search  and 
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comparison  task  was  about  40  msec  per  item  in  the  visual 
display,  plus  a  constant. 

More  recently  Schneider  and  Shiffrin  (1977)^  presented 
a  hybrid  search  task  which  included  components  of  both 
visual  and  memory  search.  Essentially  this  consisted  of 
presenting  the  input  items  in  sets  of  1 ,  2,  or  4  rather  than 
single  items  in  succession.  Additionally  subjects  were  given 
a  memory  set  size  of  1 ,  2,  or  4  items.  Thus  at  the  extremes, 
a  subject  may  be  given  a  single  input  item  to  be  compared 
with  a  single  memory  set  item,  or  simultaneous  presentation 
of  four  input  items  with  four  items  in  the  memory  set 
constituting  a  total  of  16  comparisons  (assuming  the  search 
is  exhaustive).  If  the  comparisons  were  made  in  a  serial 
fashion,  then  RT  would  be  expected  to  be  a  joint  function  of 
the  memory  set  size,  and  the  display  set  size  as  shown  in 
Figure  14,  Corrected  for  the  intercept,  RT  would  be  40  msec 
per  comparison.  This  is  essentially  what  Schneider  and 
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Figure  14.  Reaction  Time  as  a  Function  of  the 
Memory  Set  and  the  Display  Set 

( 
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Shiffrin  had  found,  although  their  actual  data  was  in  a 

more  complex  form.  Similar  findings  were  reported  by 

48 

Briggs  and  Johnsen  (1973) • 

Of  additional  interest  is  the  use  of  accuracy  as 
opposed  to  RT.  One  would  surmise  that  if  the  serial  search 
and  comparison  is  a  process  which  consumes  time,  that 
accelerating  the  pace  with  which  the  display  sets  are  pre¬ 
sented  would  interrupt  the  process  and  depress  accuracy. 
Further,  the  presentation  rate  would  interact  with  memory 
and  display  set  sizes.  This  was  essentially  the  findings 
Schneider  and  Shiffrin  reported  in  experiment  1 . 

48 

Both  Griggs  and  Johnsen  (1973)  and  Schneider  and 
Shiffrin  (1977)^  report  that  under  certain  conditions, 
extensive  practice  reduces  the  slope  of  the  RT-comparison 
functions  to  almost  zero,  i.e.  the  RT  required  for  search 
and  comparison  was  nearly  a  constant  for  varying  sizes  of 
the  memory  set  and  display  set.  One  of  the  requirements 
for  this  phenomenon  was,  of  course,  the  extensive  practice, 
while  the  second  was  that  the  subject  be  trained  under  what 
Schneider  and  Shiffrin  termed  consistent  mapping  (CM) 
conditions  as  opposed  to  varied  mapping  (VM)  conditions. 

The  CM  training  conditions  are  those  in  which  the  type  of 
items  (e.g.  numbers)  that  are  designated  as  targets,  never 
become  distractors  while  the  type  of  items  designated  as 
dis tractors  (e.g.  letters)  are  not  used  as  targets.  Under 
VM  training  conditions,  items  used  as  targets  on  some  trials 
may  be  distractors  on  other  trials.  Schneider  and  Shiffrin 
contend  that  under  CM  conditions  the  processes  become  ones 
of  automatic  detection  and  automatic  search.  Under  VM 
conditions,  the  slower,  serial,  set-size  dependent  processing 
that  obtains  is  referred  to  as  controlled  search. 
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As  LaBerge  (1976)^*  Schneider  and  Shiffrin  (1977)^ 
devote  much  of  their  effort  to  establishing  a  qualitative 
as  well  as  a  quantitative  difference  between  controlled 
and  automatic  processing.  The  first  difference  has  already- 
been  noted,  that  automatic  search  is  not  affected  by  load. 
Accuracy  and  RT  under  CM  training  conditions  are  relatively 
constant  when  the  size  of  either  the  memory  sets  or  dis- 
tractor  sets  are  increased.  Thus  automatic  search  becomes 
an  emerging  goal  of  instruction  as  the  automaticity  would 
be  a  valuable  asset  in  time-sharing  tasks  in  that  it  would 
require  minimum  resources.  A  second  difference  is  that  in 
controlled  processing,  if  a  second  target  follows  the  first 
in  close  temporal  proximity,  the  second  target  is  frequently 
not  detected,  presumably  because  the  first  is  still  being 
processed.  But  in  automatic  processing  only  the  simul¬ 
taneous  presentation  of  two  targets  produced  a  slight 
decrement.  Further,  the  two  targets  being  identical 
facilitated  the  performance  of  controlled  processing  while 
degrading  automatic  detection,  a  finding  which  the  authors 
contend  shows  support  for  the  notion  that  automaticity  is 
qualitatively  different  them  controlled  processing,  A  third 
difference  is  reported  by  Shiffrin  and  Schneider  (1977)^ 
regarding  the  rigidity  of  the  automatic  processing.  They 
report  that  the  reversal  of  the  target/distractor  status 
of  the  items  has  minimal  effect  on  post-reversal  learning 
under  controlled  processing.  However,  if  the  original 
target/distraction  distinctions  were  trained  extensively 
under  CM  conditions,  then  subsequent  reversal  training  is 
extremely  difficult*  the  idea  being  that  once  the  auto¬ 
matic  detection  processes  are  developed,  they  are  difficult 
to  modify  or  reverse. 
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A  fourth  difference  between  automatic  and  controlled 
processes,  comes  from  focused  attention  experiments.  In 
these  paradigms,  subjects  are  presented  with  some  form  of 
a  divided  display.  They  are  to  focus  their  attention 
(allocate  all  their  resources)  on  one  part  of  the  display 
while  ignoring  the  others .  Both  targets  and  distractors 
may  be  displayed  in  the  to-be-attended  portion  of  the  dis¬ 
play.  All  inputs  in  the  to-be-ignored  portion  of  the  dis¬ 
play  are  referred  to  as  foils .  Typically  performance  on 
the  attended  portion  of  the  displays  are  effected  by  the 
foils.  When  the  foils  bause  a  drop  in  performance,  this  is 
referred  to  as  a  focused  attention  deficit.  Within  the 
present  context,  focused  attention  deficits  imply  resource 

allocation  rigidity.  It  was  shown  by  Shiffrin  and 
47 

Sohneider  that  foils  which  were  targets  established  under 
CM  conditions  caused  pronounced  focused  attention  deficits 
whereas  VM  trained  foils  did  not.  It  was  argued  that  once 
an  automatic  detection  mechanism  is  established,  any  input 
(in  any  portion  of  the  display)  which  triggers  the  mechanism 
will  cause  it  to  run  its  full  course.  The  mechanism's 
status  as  automatic,  implies  that  it  is  not  under  the  con¬ 
scious  control  of  the  subject.  The  VM  target  foils,  though 
they  may  be  encoded,  seem  to  be  suppressed  in  terms  of  any 
further  processing  and  are  not  a  burden  on  the  focused 
attention  performance.  Anecdotal  reports  by  the  subjects 
in  these  experiments  further  demonstrate  the  automaticity 
quality  of  these  CM  trained  targets.  Subjects  report  that 
the  targets  being  certain  letters,  seem  to  jump  out  at 
them  even  when  doing  normal  reading. 

In  the  visual  and  memory  search  studies  described 
thus  far,  the  subjects  were  not  able  to  anticipate  the 
nature  of  an  item  before  its  presentation.  In  the 
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controllers  tasks,  we  reported  that  anticipation  was 
possible  and  probable.  The  continuous  flow  of  connected 
information  on  the  display  would  be  highly  predictable  with 
experience.  Thus  the  role  of  expectation  is  important.  It 
will  be  recalled  that  LaBerge  (1973)  reported  that  RTs  to 
novel  stimuli  were  comparable  to  familiar  stimuli  when 
cued.  Klatzky  and  Smith  (1972)^  also  showed  shorter  RTs 
for  cued  items  that  would  have  been  trained  in  VM  conditions, 
LaBerge  felt  that  the  role  of  expectancy  would  impact  the 
process  in  the  encoding  stage  by  placing  the  encoding  hier¬ 
archy  for  a  particular  event  in  an  excitatory  state. 

Sternberg  (1975)"*°  felt  that  it  was  the  comparison  process 
which  would  be  speeded  for  an  expected  item.  Shiffrin  and 
Schneider  (197^)'J  also  note  that  the  major  role  of  the 
expectancy  could  be  in  the  response  production  stage.  At 
this  point,  whether  expectancy  affects  the  encoding  stage, 
serial  comparison,  decision  making,  the  response  production 
stage  or  possibly  all  stages,  is  not  clear. 

Just  how  much  of  a  role  automaticity  and  expectation 
might  play  in  a  controller's  search  task  is  not  readily 
apparent  either.  Introspective  reports  on  the  GCA-CTS 
indicate  that  there  are  certain  "things"  that  a  novice  keeps 
"looking  for"  in  the  display  during  an  approach.  The 
novice  reports  that  in  a  sense  he  cycles  through  a " check¬ 
list"  of  cues  for  which  he  scans  the  display  (elevation 
intersect,  elevation  trail,  range  hashmark,  azimuth  intersect, 
etc.).  This  check-list,  however  it  might  be  verbalized  or 
formulated,  acts  similar  to  tha  memory  set  in  memory  search 
studies.  Further,  the  display  itself  presents  the  novice 
with  a  continuous  flow  of  multiple  inputs  requisite  for  the 
visual  search  portion  of  the  task.  While  the  novice  is 
cognizant  of  some  kind  of  a  search  process,  the  expert  is  not. 
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Does  this  mean  the  expert  engages  in  some  kind  of  automated 
search,  or  do  his  expectations  lead  him  to  a  cyclic  rhythm 
of  controlled  processing  (or  both)? 

/ 

The  12-14  hours  of  PAR  training  the  students  receive 
could  constitute  enough  trials  (under  CM  conditions)  for 
some  automatic  processing  to  begin  to  occur.  It  was  esti¬ 
mated  that  during  their  training  time,  they  would  have 
made  approximately:  42-54  calls  per  approach,  5-8  approaches 
per  session,  and  12-16  sessions  in  total,  constituting 
roughly  2,500  to  7*700  calls  made  during  training.  Shiffrin 
and  Schneider  (1977)"*^  in  experiment  1  report  automaticity 
within  roughly  the  first  2,100  trials  of  CM  training,  with 
600  trials  per  day.  However,  the  memory  set  was  well  de¬ 
fined  (simply  a  set  of  characters)  as  was  the  display  set. 
The  novice  controller  reports  "checking  for  things"  but 
the  exact  internalized  form  of  the  "things"  is  not  apparent 
nor  is  it  obvious  that  this  subjectively  contrived  memory 
set  to  which  the  novice  is  referring,  is  consistent  across 
trials,  a  condition  necessary  for  the  consistent  mapping 
requisite  for  automaticity.  In  fact,  the  student  is  prob¬ 
ably  reorganizing  his  internal  structure  at  various  junc¬ 
tures  in  his  training.  Thus  the  PAR  ins tine tor  may  be 
right  when  he  feels  that  his  students  are  not  operating 
with  automaticity  following  only  12-16  hours  actually  on 
the  display,  whereas  the  expert's  performance  after  1-2 
years  as  controller  may  be.  There  would  be  several 
possible  ways  of  testing  the  automaticity  of  the  12-16 
hour  novice,  and  the  expert  if  developed.  Some  analogue 
to  the  Stroop  Color-Word  Interference  Test  (1935)^^ 
could  be  contrived  in  which  the  expert  ^uid  novice  could 
be  compared  on  a  task  requiring  them  to  make  incompatible 
verbal  responses  to  the  events  displayed  on  the  scope. 
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In  this  case,  automaticity ,  to  the  extent  present,  would 
work  against  the  expert,  possibly  degrading  his  performance 
more  severely  than  the  controlled  processing  novice  (see 
Keele,  1972^  Eriksen  and  Eriksen,  197^'*'*  for  examples). 

PROCESS  VS.  STRUCTURE.  At  this  point  we  would  like  to  make 
a  distinction  between  process  and  structure,  although  the 
state-of-the-art  is  such  that  a  rigid  delineation  between 
the  two  concepts  is  not  always  possible.  Ve  will  take 
structure  to  refer  to  the  internal  organization  of  units 
of  a  knowledge  domain.  As  an  example,  in  the  memory  search 
tasks  of  Sternberg's,  the  list  of  items  in  memory  through 
which  the  subject  must  search,  constitutes  a  simple  structure. 
The  search  activity  itself  would  be  an  example  of  what  we 
will  refer  to  as  process .  It  is  obvious  that  errors  may  be 
caused  by  deficiencies  in  either  process  or  structure.  If 
the  search  process  itself  is  interrupted  prematurely  by 
perhaps  incoming  distractors,  a  match  may  go  undetected.  If 
one  of  the  members  of  the  memory  set  were  inadvertantly  lost, 
again  a  match  may  go  undetected.  The  point  is  that  both 
process  and  structure  could  lead  to  the  same  error —  a 
missed  target. 

It  would  be  important  for  an  intelligent  training  system 
to  be  able  to  diagnose  the  causes  of  all  the  various  types 
of  errors  that  a  student  could  make  so  that  each  could  be 
rectified  in  the  most  expeditious  manner.  However  at  this 
point,  we  will  be  content  with  a  simple  binary  diagnosis 
of  the  errors  being  either  process  or  structurally  preci- 
pated.  One  way  the  system  could  diagnose  the  "missed- 
target”  errors  in  the  present  example  would  be  to  vary 
resource  allocations.  Process  deficiency  precipitated 
errors  sould  be  a  function  of  resource  allocations  and 
would  thereby  be  termed  errors  of  resource-limitations. 
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Errors  due  to  structural  deficiencies  however,  would  appear 
to  be  data— limited  in  that  large  increases  in  resource 
investments  would  not  reduce  the  amount  of  information  or 
potential  distractors  in  other  parts  of  a  complex  display 
so  that  the  student  could  "concentrate"  on  his  memory 
search  process.  Such  increases  in  concentration  on  the 
search  process  should  increase  performance  providing  the 
problem  is  not  structural.  If  it  is,  say  an  item  is  missing 
from  the  memory  set,  then  no  amount  of  increase  in  concen¬ 
tration  (resources)  will  improve  performance. 

The  importance  of  the  process-structure  distinction 
lies  in  differential  instructional  strategies.  In  a  gross 
sense,  increased  efficiency  in  processing  comes  with  practice 
while  structure  is  modified  and  built  through  instruction  and 
the  imparting  of  knowledge.  In  our  memory  search  example, 
if  the  system  had  determined  that  the  misses  were  due  to 
resource-limitations,  it  would  prescribe  additional  practice 
on  that  part  of  the  process  until  it  becomes  more  automatic 
and  requires  less  resources.  If  however,  the  system  had 
determined  that  the  misses  were  due  to  data-limitations,  it 
could  stop  and  query  the  student  on  structure  (memory  set). 
When  it  found  that  part  of  the  memory  set  was  missing,  it 
would  proceed  to  rectify  the  situation  through  tutorials 
as  to  the  proper  memory  set  and  then  return  to  the  main 
portion  of  the  training  system. 

Our  distinction  between  process  and  structure  is  not 

as  clear  as  we  would  like  however,  Empirical  evidence 
/  48 

(Briggs  and  Johnsen,  1973;  and  Schneider  and  Shiffrin, 

47  \ 

1977  )  shows  us  that  processing  becomes  more  efficient 

(requires  less  resources)  even  to  the  point  of  automaticity, 
possibly,  with  extensive  practice.  It  is  not  apparent, 
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however,  as  to  what  exactly  has  changed  ±n  the  processing 
that  makes  it  faster  and  more  automatic.  This  is  especially 
so  in  a  more  complex  situation  such  as  a  controller.  Here 
there  is  more  than  a  simple  memory  set  to  be  examined  wherein 
a  "match”  leads  to  a  simple  "detection"  response.  The 
unspecified  units  to  be  searched,  rather  than  being  in  a 
list  form  may  be  related  in  various  ways  forming  a  complex 
network.  Further,  once  a  "match"  is  found,  additional  pro¬ 
cessing  may  be  required  in  order  to  construct  the  correct 
form  of  the  response.  With  extensive  practice  can  we  say 
that  the  search  through  the  complex  network,  and  the  construc¬ 
tion  of  a  response  becomes  automatic,  so  that  it  may  be 
carried  out  in  parallel,  (i.e.  the  same  number  of  steps 
are  required  but  are  now  processed  without  effort)?  Or 
may  we  say  that  with  extensive  practice  certain  steps  may  be 
eliminated,  or  the  organizational  structure  is  somehow  modi¬ 
fied  and  streamlined,  so  that  the  input  data  leads  more 
directly  to  the  response?  To  this  end  we  need  to  look  at 
structure,  the  problem  of  how  knowledge  is  internally 
represented,  and  the  process  by  which  detection  of  some 
event  ultimately  leads  to  the  selection  of  a  response. 

There  has  been  much  research  effort  of  late  which 
has  been  directed  toward  this  problem  of  internal  repre¬ 
sentation  and  structure,  a  good  portion  of  which  has  been 
funded  by  ONR.  Several  models  of  interest  have  been  pro¬ 
posed  by  Anderson  (1978)^  Collins  and  Loftus  (1975)^ 
Hayes-Roth  (1977)^  Kintsch  (1972)^  and  Norman,  Rumelhart, 
and  the  LNR  research  group  (1975)^  The  terminology  differs 
somewhat  but  most  of  the  models  are  derivatives  of  what 
was  termed  Associative  Network  Models. 


69 


NAVTRAEQUIPCEN  79-C-0085-1 


Most  current  models  assume  that  knowledge  exists  In 
memory  In  conceptual  chunks.  These  chunks,  on  cognitive 
structural  units  have  been  described  variously  as  records 
(Norman  and  Bobrov,  1979**1)  schemata  (Anderson,  1978^ 

Bobrow  and  Norman,  1975^2)  cogits  (Hayes-Roth,  1977^)* 
frames  (Minksy,  1975  J) ,  and  nodes  (numerous  authors). 

In  their  usage,  these  terms  are  in  no  way  synonyms,  but 
all  refer  in  an  abstract  way  to  separate  knowledge  enti¬ 
ties.  In  attempting  a  short  amalgamation  of  the  concepts 
of  the  various  authors,  the  terminology  presented  by  Norman, 
Bobrow,  and  their  colleagues  is  relied  upon  heavily,  due 
to  the  continuity  of  their  efforts  from  about  1975.  Some 
variations  in  the  usage  of  terms  arise  however,  when  attemp¬ 
ting  to  incorporate  principles  from  other  authors.  Readers 
interested  in  more  detail  in  these  areas  are  encouraged  to 
examine  the  original  sources . 

I 

The  "vague"  chunks  of  memory  will  be  referred  to  as 
schemata,  a  term  used  in  various  ways  by  several  authors. 

By  this  we  will  mean  a  segment  of  knowledge  organized  around 
a  central  concept  or  theme.  An  example  of  a  schema  in  its 
simplest  form  would  be  the  frame-like  notation  presented 
by  Minsky  (1975)  J  for  representing  knowledge  in  the  com¬ 
puter.  Here  a  "frame"  is  a  description  of  an  object,  or 
action,  which  incorporates  all  the  invariant  features  com¬ 
mon  to  instances  which  would  be  classed  as  an  example  of 
the  particular  abject  or  action.  Kuipers  (I975)f**  discusses 
and  lists  the  properties  of  frames,  while  Bobrow  and  Norman 
(1975)  and  Vinograd  (1975)  0  discuss  some  of  the  under¬ 
lying  issues  leading  to  more  recent  modifications  of 
Minsky" s  ideas. 
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An  example  of  a  frame-like  schema  is  given  by  Bott 

(1979)^  and  is  shown  in  Figure  15.  The  schema  is  similar 

in  a  way  to  the  older  ide&  of  concepts  (see  Trabasso  and 
67 

Bower,  1968  ')  in  that  the  schema  is  seen  as  a  knowledge 

DOG 

Head 

Body 

Tail 

Color 

Location 

Breed 

Age 

Name 

to  find  name:  examine  dog  collar  (procedure) 
Owner 

to  find  owner:  examine  dog  collar  (procedure) 
Figure  1 5 •  Exemplar  Schema  for  the  Concept  Dog 

unit  comprised  of  slots  for  the  attributes  or  features  by 
which  it  is  defined.  The  schema  is  more  than  this  however, 
in  that  the  slots  may  contain  subparts,  supersets,  and  impor 
tant  in  the  present  context,  procedural  information. 

Bo  brow  anr1  Norman  (1975)  make  the  assertion  that 
schemata  in?  comprised  of  context-dependent  descriptions. 

They  proposed  that  the  descriptions  were  formed  by  selecting 
a  set  of  descriptors  which  would  unambiguously  set  apart  the 
object  or  event  in  question  from  other  referents  present  in 
the  current  context  at  the  time  of  the  description's  first 
usage.  If  a  child  first  forms  an  internal  representation 
of  a  "dog"  in  the  context  of  a  farm  with  much  larger  animals 
then  his  Set  of  descriptors  would  differ  somewhat  from  the 
formation  of  a  description  in  a  different  context.  New 


NAVTRAEQUIPCEN  79-C-0085-1 


experiences,  such  as  subsequent  encounters  with  "dog-like" 
organisms  are  interpreted  and  classed  by  the  original 
context-dependent  descriptions.  Thus  the  original  encoun¬ 
ters  serve  as  prototypes  for  subsequent  encounters  (Norman 
and  Bobrov,  1979^*)* 

The  function  of  schemata  is  not  only  to  aid  the  person 
in  functioning  in  an  environment  in  which  he  is  already 
experienced,  but  also  in  novel  situations.  In  a  new  situa¬ 
tion  wherein  the  input  does  not  match  existing  schemata,  a 
search  is  made  for  a  "near"  fit  in  an  analogical  or  meta¬ 
phorical  sense  by  selecting  perhaps  a  schema  from  a  different 
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context.  Norman,  Gentner  and  Stevens  (1976)  present  an 
example  of  this  in  their  Mayonnaise  Problem.  Subjects  were 
asked  their  opinion  regarding  the  probable  ingredients  of 
mayonnaise.  Being  a  white  creamy  substance,  the  naive 
subjects  often  responded  with  dairy  products,  the  result 
of  existing  schemata.  Nowhere  in  their  existing  schemata 
could  the  results  of  the  interaction  between  egg  yolks  and 
oil  producing  a  white  creamy  substance,  be  found.  Thus, 
confronted  with  this  new  problem,  the  subjects  selected  a 
schema  by  analogy  and  then  proceeded  to  modify  the  struc¬ 
ture  until  a  solution  could  be  output. 

Of  present  interest  regarding  the  training  of  radar 

controllers,  is  the  relation  between  schemata  and  resource 

consuming  events.  Assume  for  the-  moment  that  schemata  may 

be  both  event-driven,  being  a  "bottom-up"  (Bobrow  and  Norman, 
62 

1975  )  process,  or  conceptually-driven  as  a  "top-down" 

process  (Goodman,  1976^^).  In  an  event-driven  situation 
as  described  in  the  previous  section,  an  input  generates 
a  description  which  must  be  compared  with  potential  schemata. 
Early  in  training  this  search  and  comparison  process  (if 
that  is  all  it  is)  consumes  both  time  and  resources.  If  a 
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quick  match  is  found,  the  input  is  fit  into  a  context  which 
may  provide  a  further  input  to  a  higher  level  schema,  or 
may  lead  directly  to  procedural  information  for  response 
production.  If  a  match  is  not  found  however,  the  central 
resources  are  then  required  to  examine  the  degree  of  mis¬ 
match  in  the  various  comparisons.  The  mayonnaise  problem 
would  be  an  extreme  mismatch  problem  in  that  considerable 
resources  and  time  would  be  required  to  generate  a  response. 
Even  with  no  other  concurrent  tasks  and  unlimited  time, 
the  students  could  not  give  the  correct  ingredients, 
which  would  make  the  problem  a  data-limited  (or  memory- 
limited)  task  caused  by  structural  deficiencies.  Note  that 
mismatches  do  not  preclude  responses  from  occuring.  If 
however,  the  degree  of  mismatch  is  slight,  the  central 
mechanism,  having  access  to  a  wide  array  of  schemata,  may 
be  able  to  resolve  the  mismatch  leading  eventually  to  the 
appropriate  response.  Unavailability  of  the  central  mech¬ 
anism  or  severe  time  constraints  would  prohibit  responding 
or  cause  the  central  mechanism  to  execute  some  default 
option  in  its  output  which  would  very  likely  be  an  error. 

One  cause  for  resource  and  time  consumption  then 
would  be  the  occurence  of  mismatches,  which  speaks  of  the 
importance  of  the  development  of  adequate  descriptions  and 
prototypes  during  training.  A  second  possible  relationship 
between  resource  and  time  consumption  and  structure  could 

Kg 

be  in  the  strength  of  the  structure.  Hayes-Roth  (l977r 
proposing  a  slightly  different  network  model,  posits  that 
a  knowledge  structure  changes  in  strength  as  well  as,  pos¬ 
sibly  configurations.  In  her  treatise,  the  smallest  cog¬ 
nitive  units  of  information  are  called  cogits.  The  cogits, 
activated  in  an  all-or-none  fashion,  are  assembled  via 
associations  into  larger  configurations.  A  particular 
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configuration  is  said  to  have  "strength"  as  a  function  of 
the  strengths  of  the  separate  associations  between  cogits. 
With  extensive  practice  the  assembly  can  eventually  be 
strengthened  to  the  point  of  "unitization"  whereupon  it 
may  be  activated  on  in  a  discrete  all-or-hone  fashion. 
Hayes-Roth  and  Hayes-Roth  ( 1 975 ) ;  and  Perlmutter,  Sorce, 
and  Myers  (1976) 71  both  report  reaction  times  in  verifying 
a  learned  proposition  decreases  with  practice.  Thus  for 
training  applications,  mere  practice  and  repetition  may  not 
only  effect  process  but  also  the  strength  of  a  structure. 
However,  here  the  distinction  between  process  and  structure 
blurrs  somewhat  in  that  the  strength  is  inferred  by  acti¬ 
vation  speed  which  would  be  process. 

Expectation  can  be  thought  of  as  the  activation  of  a 
schema,  or  series  of  structures  in  a  top-down  sense,  just 
prior  to  the  next  input.  As  an  example,  suppose  that  an 
LSO,  tracking  an  approaching  aircraft  just  above  the  hori¬ 
zon,  notices  a  puff  of  smoke  from  the  tail.  This  additional 
exhaust  emission  would  result  from  the  pilot  applying  more 
power.  The  visual  input  in  that  context  would  activate  the 
appropriate  schema  corresponding  to  the  increase  in  power. 

In  addition  to  any  procedural  component  activated  in  the 
schema,  the  schema  could  provide  conceptual  input  into 
another  schema  corresponding  to  gains  in  altitude.  Thus 
the  first  schema  was  data-driven  while  the  second  was  con¬ 
ceptually  driven.  Having  just  detected  a  puff  of  smoke, 
he  now  "expects"  to  see  a  gain  in  altitude.  The  reader 
may  want  to  refer  to  Anderson  (1978)56  regarding  a  discus¬ 
sion  of  data-driven  and  conceptually-driven  processes  in  the 
act  of  reading. 

Rumelhart  and  Norman  (1976)72  and  Norman  (1978)73 
propose  that  the  learning  or  formation  of  the  cognitive 
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structures  occur  in  three  overlapping  stages.  The  first 
stage  which  they  called  accretion  of  knowledge,  most  gen¬ 
erally  refers  to  the  accumulation  of  new  information  and 
facts  in  accordance  with  existing  knowledge.  During  this 
stage,  the  instructional  system  is  adding  to  the  underlying 
data  base  of  the  learner.  Incoming  information  is  organized 
via  existing  knowledge  structure.  Thus  a  student  with  prior 
experience  in  a  different  type  of  air^  traffic  control  would 
find  additional  control  duties  easy  to  Acquire  because  he 
had  extensive  structures  in  this  area  already  formulated. 

A  complete  novice  however,  may  be  forced  to  generate  struc¬ 
ture  from  scratch  as  it  were.  As  an  example,  the  novice, 
having  had  no  previous  schemata  established  regarding  radar 
technology,  must  use  his  existing  knowledge  to  understand 
how  it  works.  Choosing  a  prototype  from  another  context, 
he  may  decide  that  radar  must  be  something  like  photography. 

So  he  begins  to  establish  his  conceptualization  of  radar 
around  what  he  knows  of  photography.  As  one  might  guess, 
the  student  will  encounter  various  situations  where  this 
analogy  will  break  down. 

The  second  stage  is  referred  to  as  re s true tur ing .  Vhen 
the  existing  schemata  cannot  handle  the  new  knowledge,  the 
schemata  themselves  may  need  to  be  reorganized.  It  is  the 
restructuring  of  knowledge,  rather  than  the  addition  of  new 
facts,  which  leads  to  the  kind  of  phenomena  we  call  "insight." 
The  novice  may  have  television  and  radio  transmissions,  visual 
sensations,  auditory  sensations,  radar  return,  and  photography 
all  compartmentalized  as  entirely  different  entities.  But  if 
questioned  about  the  similarities  in  the  type  of  signals 
transmitted  and  received,  the  student  m^y  come  to  conceptu¬ 
alize  light,  sound,  radio  transmissions,  radar  return,  and 
even  heat  as  basically  the  same  thing,  differing  only  in 
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wave-length  or  frequency.  The  type  of  training  to  produce 

this  phenomena,  would  probably  be  that  of  a  "Socratic  tutor" 
/  74\ 

(see  Stevens  and  Collins,  1977  )  which  would  pick  at  the 

student's  present  knowledge  structures  exposing  the  inade¬ 
quacies  of  his  present  schemata,  so  that  they  might  be 
modified.  The  restructuring  stage  we  would  associate  most 
with  the  concept  of  understanding. 

Of  special  interest  in  the  training  of  controllers  is 
the  third  stage  referred  to  as  tuning.  Once  the  new  know¬ 
ledge  has  been  added  to  the  data-base,  perhaps  causing 
some  restructuring  of  the  existing  schemata,  the  schemata 
undergo  additional  minor  changes  depicted  as  "fine  tuning." 
Here  the  processing  associated  with  schemata  undergoing 
tuning  becomes  streamlined,  more  efficient,  requires  less 

resources  and  eventually  reaches  a  stage  of  automation. 

7  2 

Rummelhart  and  Norman  (1976)  suggest  that  in  tuning,  the 
structure  remains  unchanged  with  only  the  constant  and  vari¬ 
able  terms  to  which  they  refer  undergoing  revision.  These 
terms  can  be  changed  by 5  improving  their  accuracy,  extend¬ 
ing  the  range  of  the  variables  to  generalize  the  applica¬ 
bility,  constraining  the  variables  to  make  the  applicability 
more  specialized,  and  the  assumption  of  default  values  for 
certain  variables  when  the  variables  go  unspecified.  This 
last  modification,  the  assumption  of  default  values,  may 
be  an  important  feature  of  a  finely  tuned  schemata  during 
the  processing  of  a  highly  speeded  task. 

Norman  (1978)^  views  these  three  modes  of  learning 
as  occurring  essentially  in  the  sequence  of  accretion 
first,  then  restructuring,  and  finally  tuning  although  the 
sequence  is  not  one  of  discrete  stages..  All  modes  are 
probably  present  throughout  training  but  differ  in  tie 
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degree  to  which  they  are  occurring.  In  general,  the  instruc¬ 
tional  prescriptions  for  the  three  inodes  of  learning  respec¬ 
tively  would  be: 

(1)  presentation  of  information  and  facts  in  lecture, 
textual  material,  etc. 

(2)  the  posing  of  problems  and  questions  designed  to 
expose  structural  deficiencies,  and  finally 

(3)  extensive  practice  for  the  necessary  tuning. 

RESPONSE  EXECUTION.  There  are  currently  two  major  theories 
that  dominate  and  generate  research  in  the  area  of  the 
learning  and  retention  of  motor  skill.  These  are  Adams’ 
(1971)^'*  closed-loop  theory,  and  Schmidt’s  ( 1 975)^schema 
theory  of  motor  behavior.  The  two  are  fairly  consistent 
when  considering  the  effects  and  importance  of  certain 
variables  and  dimensions,  but  they  do  differ  in  important 
ways.  Each  theory  reflects  the  current  Zeitgeist  to  develop 
analytical  systems  in  terms  of  information  processing  con- 
siderations.  Being  the  earlier  account,  Adams’  (1971) 
approach  retains  vestigial  nomenclature  from  the  period  of 
S-R  domination  of  accounts  of  human  behavior.  Schmidt's 
(1975)'  theory  relies  less  heavily  on  S-R  conceptuali¬ 
zation,  but  none-the-less ,  may  be  easily  related  to  Adams'. 

Adams'  theory  contains  all  the  requisites  for  it  to 
be  classified  as  being  closed-loop.  Specified  by  the  theory 
is  the  development  of  a  reference  mechanism  for  movement. 

The  reference  mechanism  is  defined  by  the  occurences  of 
response -produced  stimuli  during  the  learning  of  the  criterion 
response,  and  comes  to  be  an  internal  standard  for  correct¬ 
ness.  Subsequent  feedback  from  a  response  is  compared  with 
the  reference  and  discrepancies  are  recognized  as  errors  to 
be  corrected.  The  theory  is  centrally  concerned  with  the 
feedback  mechanisms  that  are  involved  in  the  learning  and 
retention  processes. 
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Tlie  reference  mechanism  developed  during  acquisition  is 
called  the  perceptual  trace  and  it  is  a  function  of  all  the 
stimuli  associated  with  the  response.  Any  sensory  modality 
acting  at  the  time  of  acquisition  contributes  response- 
produced  stimuli  that  are  a  part  of  the  resulting  perceptual 
trace.  The  strength  of  the  perceptual  trace  is  a  positive 
function  of  the  number  of  times  the  stimuli  from  which  it 
is  developed  are  experienced. 

The  perceptual  trace  can  be  viewed  as  a  version  of  the 
image  or  representation  in  perception.  A  position  mentioned 
previously  in  visual  perception  is  that  one  recognizes  objects 
when  their  stimuli  arouse  and  match  a  perceptual  internal 
representation  acquired  from  prior  experience.  In  a  like 
manner,  a  subject  can  recognize  a  response  that  he  has  made 
before  when  feedback  stimuli  arouse  and  match  the  perceptual 
trace  of  that  response.  Knowing  the  correctness  of  a  res¬ 
ponse  is  a  matter  of  response  recognition  governed  by  the 
comparison  between  current  response  produced  stimuli  and  the 
perceptual  trace.  If  an  insufficient  match  is  made  and  an 
error  is  detected,  the  response-generating  mechanism  can 
deliver  a  new  response  which  in  turn  is  subjected  to  the 
comparison  process.  This  continues  until  a  successful  match 
occurs  or  until  the  system  is  exhausted  of  potential  responses 

The  response-generating  mechanism  is  called  the  memory 
trace  and  it  governs  the  selection  of  responses  to  be  sub¬ 
jected  to  the  comparison  process.  The  memory  trace  is  seen 
as  the  associative  agent  responsible  for  the  activation  of 
a  response.  The  memory  trace  is  important  because  it  is  the 
process  which  activates  the  response  in  the  first  place  with 
the  perceptual  trace  governing  the  extent  of  the  movement. 

It  is  necessary  to  postulate  the  existence  of  this  second 
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memory  component  because  there  must  be  independent  agents 
governing  the  elicitation  and  comparison  processes.  If  one 
agent  were  responsible  for  both,  the  response  would  neces¬ 
sarily  be  checked  against  itself  and  no  errors  would  be 
detected.  The  memory  trace  may  be  viewed  as  a  modest  motor 
program  which  only  governs  the  choice  of  responses,  not 
extent  or  duration  as  other  conceptions  of  a  motor  program 
might  have  it.  Moreover,  there  is  a  sufficient  body  of 
evidence  suggesting  that  verbal  recall  and  recognition  re¬ 
flect  different  memory  processes  wherein  recall  is  seen  as 
response  production  and  recognition  as  response  identifica¬ 
tion  (although  these  may  not  always  be  mutually  exclusive; 
Marshall  and  Smith,  1977^).  Parsimony  would  require  the 
same  division  of  motor  memory.  The  two  mechanisms  of  per¬ 
ceptual  and  memory  trace  are  seen  as  the  basis  for  recog¬ 
nition  and  recall,  respectively. 

According  to  Adams'  theory,  forgetting  can  be  accounted 
for  by  a  deterioration  both  of  memory  and  perceptual  traces. 

A  response  produced  by  a  memory  trace  weakened  by  forgetting 
will  induce  the  selection  of  inappropriate  movements. 
Deterioration  of  the  perceptual  trace  will  result  in  a  poor 
reference  mechanism  and,  even  if  the  memory  trace  initiates 
a  correct  response,  the  reference  mechanism  may  be  so  deteri¬ 
orated  as  to  make  *  match  impossible.  The  theory  as  presently 
stated  says  nothing  about  the  mechanisms  of  forgetting,  only 
about  the  results  of  the  operations  of  these  mechanisms. 

Major  mechanisms  that  have  beer  postulated  are  trace  decay 
and  interference .  The  learning  and  retention  of  a  movement 
always  occur  in  the  presence  of  time  (trace  decay),  and  the 
interaction  with  other  movements  and  their  perceptual  traces 
(interference).  Whether  the  memory  trace  and  perceptual 
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traces  undergo  a  spontaneous  decay,  are  effected  by  inter¬ 
fering  stimuli,  or  both,  is  an  issue  not  decided  by  the 
theory.  However,  a  comprehensive  theory  of  learning  in 
general,  and  motor  learning  in  particular,  must  incorporate 
some  mechanisms  for  forgetting. 

7  8 

In  one  study,  Adams,  Goetz,  and  Marshall  (1972)' 
were  concerned  about  the  role  of  response-produced  feed¬ 
back  during  acquisition  trials  when  knowledge  of  results 
(KR)  is  delivered  after  each  response  and  on  trials  where 
KR  is  withdrawn.  The  data  supported  the'  theory  in  several 
ways : 

(1)  Performance  in  acquisition  was  positively  re¬ 
lated  to  amount  of  proprioceptive  feedback; 

(2)  Performance  on  KR  withdrawal  was  better  with 
augmented  than  with  impoverished  feedback; 

(3)  Performance  was  positively  related  to  amount  of 
practice  and  strength  of  the  perceptual  trace; 

(4)  Performance  during  KR  withdrawal  was  determined 
by  similarity  of  feedback  during  acquisition  and 
KR  withdrawal. 

Marshall  (1972)  y  in  a  study  comparing  recognition 
and  recall  in  short-term  memory,  concluded  that  "Though 
the  two  measures  required  different  behavior  on  the  part 
of  the  subject,  the  closed-loop  theory  would  view  them  both 
as  being  determined  by  the  state  of  the  'perceptual  trace  at 
the  time  of  test,  and  they  both  should  be  the  same  function 
of  the  same  variable"  (page  152).  The  Results  supported 
the  theoretical  assumption. 

The  Marshall  ( 1972)^study  was  the  first  to  subject 

the  notion  of  recognition  in  motor  memory  to  empirical 
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test.  Adams  and  Goetz  (1973)  made  use  of  a  similar 
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methodology  to  empirically  separate  the  error-detection 
from  the  error-correction  processes.  The  essence  of  their 
methodology  was  that  ability  to  recognize  a  test  movement 
as  being  different  from  the  previously  experienced  criterion 
movement  would  reveal  the  error  detection  capability,  while 
in  a  second  study,  an  attempt  to  correct  an  erroneous  move¬ 
ment  would  reflect  the  degree  of  error  correction  ability. 

In  their  study,  both  the  effects  of  feedback  and  KR  trials 
were  potent  variables,  as  closed-loop  theory  suggests. 

While  there  are  sufficient  data  to  substantiate  the 
existence  of  error  detection  and  correction  processes  as 
postulated  by  Adams,  there  remains  the  issue  of  the  nature 
of  the  reference  mechanism  itself,  how  it  develops,  and 
how  it  operates.  Chief  among  the  critics  of  Adams'  (1971)^^ 
theory  has  been  Schmidt  (l975)»  who  published  a  seminal 
article  on  "A  Schema  Theory  of  Discrete  Motor  Skill 
Learning."  The  essence  of  Schmidt's  theory  is  based  "on 
the  notion  of  the  schema  and  uses  a  recall  memory  to  pro¬ 
duce  movement  and  a  recognition  memory  to  evaluate  response 
correctness"  (page  225).  At  first  reading  there  appears  to 
be  little  discrepancy  with  Adams,  but  one  important  dif¬ 
ference  that  has  generated  much  recent  research  concerns 
the  nature  of  the  reference  mechanism.  Instead  of  the 
development  of  an  internal  perceptual  representation, 

Schmidt  states  what  actually  develops  is  a  schema  for  the 
response  similar  to  the  schemata  reported  earlier*  A 
schema  is  defined  as  "a  characteristic  of  some  population 
of  objects,  and  consists  of  a  set  of  rules  serving  as  in¬ 
structions  for  producing  a  population  prototype  (the  con- 
cept)y  (Evans,  1967  page  87  '*  According  to  Schmidt 

(as  well  as  Posner  and  Keele,  1968f^  Bartlett,  1932?’^ 

Evans,  1 967® 1  Neumann,  197^f**1977^)  as  exemplary  stimuli 
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are  experienced  the  subject  develops  a  prototype  of  them, 
and,  depending  on  the  paradigm,  is  capable  of  recalling  or 
recognizing  the  prototype  without  ever  having  experienced 
it.  Schmidt's  schema  results  from  the  integration  of 
information  stored  about : 

(1)  the  initial  conditions, 

(2)  the  response  specifications  for  the  motor  program, 

(3)  the  sensory  consequences  of  the  response  produced, 

(4)  the  outcome  of  that  movement. 

It  is  further  assumed  by  the  theory  that  variable  input  of 
movement  may  provide  an  even  better  basis  for  the  later 
recall  of  the  prototype  movement  than  would  constant  prac¬ 
tice  to  the  criterion  movement . 

This  and  other  points  of  controversy  between  the  two 
theories  have  been  the  focus  of,  and  have  some  support  in 
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much  recent  motor  memory  research  (Kelso  and  Norman,  1978; 
McCracken  and  Stelmach,  1 977? 7  Newell,  1 973?®1 974?91 976b?0 
Newell  and  Chew,  1974?1  Newell  and  Shapiro,  1976?2  Schmidt, 
Christenson  and  Rogers,  19751^  Williams,  1978?^  Williams 
and  Rodney,  1978?"*  Zeloznik,  1977?**  Zeloznik  and  Spring, 
1976?7  Zeloznik,  Shapiro  and  Newell,  1978^®). 

The  issue  at  hand  in  this  report  concerns  the  acqui¬ 
sition  of  a  complex  motor  skill  such  as  is  represented  in 
the  AIC  task  via  an  automated  instructor  model.  The  theo¬ 
retical  counts  just  presented  place  major  emphasis  for 
accurate  performance  on  the  individual's  ability  to  monitor 
his  performance  of  errors  and  to  correct  those  errors  when 
they  occur.  One  of  the  most,  if  not  the  most,  important 
determiner  of  that  ability  during  learning  is  the  nature 
of  the  feedback  that  the  individual  receives  during  training 
Knowledge  of  results  (pnly  loosely  analagous  to  the 
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Skinnerian  concept  of  reinforcement)  is  taken  as  the 
training  vehicle  for  the  acquisition  of  this  ability.  In 
a  recent  review  of  currently  accepted  thinking  about  know¬ 
ledge  of  results  (KR)  Adams  (1978)^  reviews  four  "legacies" 
of  Thorndike  ( 191 3*  "*^1 932**^ and  1935^^)  that  are  today 
tacitly  accepted  and  explores  some  of  their  ramifications 
for  motor  learning.  These  will  be  reviewed  briefly  here 
since  they  offer  some  ramifications  for  extended  research 
on  skill  acquisition. 

The  first  of  these  is  the  observation  that  punishment 
is  hardly  ever  used  in  the  shaping  of  motor  behavior. 

Adams'  search  for  literature  concerning  the  use  of  physi¬ 
cally  aversive  KR  was  "almost  in  vain."  The  only  exception 
has  been  the  series  of  experiments  conducted  by  Payne  and 
his  associates  (Payne,  197o]0^  Payne  and  Artley,  1972;^* 
Payne  and  Richardson,  1972]^  197^]°^  and  Payne  and  Dunman, 
1974  ').  In  these  studies,  Payne  has  shown  that  a  pre¬ 

viously  neutral  stimulus,  having  been  associated  with  an 
aversive  electric  shock,  can  be  an  effective  signal  for 
off-target  behavior  in  a  tracking  task.  The  assumed  mecha¬ 
nism  is  the  fear  generated  by  that  stimulus  when  the  sub¬ 
ject  wanders  off  the  target  on  the  tracking  task.  There 
is  also  the  alternative  hypothesis,  however,  that  the 
emotional  component  merely  makes  the  signal  a  more  distinc¬ 
tive  cue . 

The  second  consideration  concerns  the  time  at  which, 
and  the  number  of  times,  KR  is  delivered.  In  the  typical 
learning  situation,  KR  is  delivered  at  the  end  of  the 
response,  and  if  this  response  is  complex,  the  entire 
sequence  must  occur  before  feedback  is  given.  In  essence, 
the  achieving  of  the  goal  is  the  event  that  initiates  KR, 
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but  the  steps  that  provide  the  means  for  achieving  that 
goal  are  typically  securing  without  the  potential  benefit 

OQ 

of  KR.  As  Adams  (1978) 77  states,  "The  price  that  we  have 
paid  for  following  this  tradition  is  that  we  know  little 
about  training  precise  movement  sequences  in  reaching  a 
precise  endpoint.  .  .  It  is  time  we  begin  thinking  about 
scenarios  of  multiple  KR  events  that  will  train  long  move¬ 
ment  sequences  in  attaining  goals"  (page  233). 

The  third  Thoxndikian  legacy  is  that  KR  is  assumed 
to  be  the  agent  responsible  for  the  strengthening  of 
"habits,"  "bonds,"  or  ’fconnections . "  In  possible  refutation 
of  this  S-R  conceptualizatL on,  Adams  considers  that  the 
motor  learning  situation  may  be  characterized  as  one  in 
which  the  individual  is  chiefly  concerned  with  detecting 
and  reducing  error.  Unlike  reinforcement,  KR  is  delivered 
in  the  hope  that  the  individual  changes  the  next  response 
so  as  to  make  it  more  reflect  the  standard.  Thus  for 
Adams,  KR  has  a  definate  motivational  component  designed 
to  bring  the  individual  into  a  more  error- free  state. 

The  final  point  concerns  the  nature  of  KR.  Knowledge 
of  results  has  been  assumed  to  be  an  objective,  external 
event,  yet  Adams  and  others  have  challenged  this  since  we 
have  the  ability  to  judge  the  correctness  of  our  own 
responses  (Adams  and  Bray,  1970;  Adams  and  Goetz,  1973? 
Newell,  1974  7).  He  argues  for  the  notion  of  "subjective 
reinforcement"  and  for  the  "fascinating  implication.  .  . 
that  after  some  learning  with  KR  has  occurred,  and  the 
power  to  appraise  our  own  error  subjectively  is  developed, 
we  should  be  able  to  learn  without  KR  because  the  error 
information  for  response  correction  is  now  available  from 
within  us,"  (page  237). 
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Human  Performance  Model.  Adams  (1971  )?"*Gentile  (  1972) ]^and 

most  recently  Marteniuk  (I976)^^have  all  made  the  observation 

that  the  study  of  skilled  behavior  has  historically  been 

piecemeal  in  nature  with  little  in  the  way  of  theory  to 

/  \75 

act  as  a  unifying  thread  for  the  research.  Adams  (1971) 
has  even  gone  so  far  as  to  nblamen  applied  research  for  this 
dilemma  since  the  emphasis  there  is  usually  on  solving  a 
particular  problem  at  a  particular  time. 

Marteniuk  has  recognized  this  absence  of  unifying 
thinking  and  in  his  recent  text  on  the  matter  (1976)^° 
he  has  attempted  to  produce  "a  unified  concept  of  how  man 
performs  and  learns  skills."  Xn  his  presentation,  he  docu¬ 
ments  not  only  the  underlying  theoretical  concepts  involved 
in  skill  learning,  but  also  a  practical  model  for  the 
teaching  of  motor  skills.  This  section  will  review  the 
essential  elements  of  the  human  performance  model  (HPM) 
described  by  Marteniuk  as  well  as  the  implications  they  have 
for  the  teaching  of  dcills. 

The  model  is  essentially  an  information  processing 
model  pertaining  to  motor  skills  that  has  as  its  major 
components:  (l)  attention  mechanisms,  (2)  perceptual  mech¬ 

anisms,  (3)  decision  mechanisms,  and  (4)  effector  mechanisms. 
The  incorporation  of  each  of  these  components  into  the  gen¬ 
eral  HPM  is  based  upon  the  "state-of-the-art"  in  each 
area.  A  reader  with  general  knowledge  of  these  areas 
should  be  able  to  follow  the  integration  of  them  and  no 
attempt  will  be  made  here  to  assess  individually  the  validity 
of  the  assumptions  about  them.  The  reader  should  be  cautioned 
however,  in  that  some  of  the  conceptual  conclusions  are  still 
in  a  state  of  evolution.  For  an  example,  the  model  assumes 
in  the  discussion  of  memory  factors,  a  multiple  storage 
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representation  of  human  memory*  This  has  been  substantially 
criticized  by  Craik  and  his  associates  (see  Craik  and 
Lockhart,  1972"^)  in  their  development  of  a  "levels  of 
processing"  framework  for  memory  research  (itself  the  tar¬ 
get  of  more  recent  criticisms!).  It  is  the  case,  however, 
that  the  final  vote  on  this  matter  is  still  to  be  taken,  and 
at  any  rate,  for  much  of  the  motor  theorizing  to  date  the 
multiple  stage  approach  still  enjoys  considerable  use. 

The  HPM  model  is  essentially  a  closed-loop  system 

with  both  extrinsic  and  intrinsic  sources  of  feedback 

affecting  its  operation.  External  feedback  is  essentially 

operationally  defined  by  KR  and  the  previous  discussions 

of  potential  innovations  in  our  thinking  about  KR  as 
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stated  by  Adams  (1978)  y  are  particularly  relevant.  The 
intrinsic  sources  of  KR  are  essentially  response-produced 
stimuli  from  any  sensory  channel  involved  in  executing  the 
movement.  The  intrinsic  sources  of  feedback  also  serve  to 
give  the  individual  information  about  the  performance  of 
the  response  while  it  is  being  executed.  While  the  extrinsic 
feedback  typically  gives  the  performer  information  as  to 
whether  the  objective  of  the  response  has  been  met. 

f 

Most  important  for  the  purposes  of  this  report  is  the 
distinction  made  by  the  model  between  open  skills  and  closed 
skills  (not  to  be  confused  with  open-loop  or  closed-loop 
theorizing) •  These  two  classes  of  skills  are  defined  in  terms 
of  the  environment  in  which  they  are  performed.  A  closed 
skill  is  performed  in  an  environment  where  the  stimuli  and 
cues  controlling  the  response  are  static,  and  tinchanging. 

‘  f 

Examples  would  be  the  execution  of  a  tee  shot  in  golf,  the 
kick-off  in  football,  or,  for  our  purposes,  the  pushing  of 
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a  sequence  of  buttons  in  response  to  a  given  display  code  in 
the  AXC  task.  The  open  skills  are  defined  as  those  in  which 
the  controlling  environment  is  continually  changing. 
Basketball,  a  kick-off  return,  or  the  tracking  of  a  constantly 
changing  visual  target  would  be  good  examples  of  open  skills. 
There  are  important  considerations  for  the  teaching  of  each 
of  these  types  and  those  considerations  will  be  presented 
in  a  later  section. 

'  An  important  consideration,  is  that  of  limited  central 
processing  capacity.  Given  that  any  task  produces  a  load 
in  the  central  nervous  system's  processing  capacity,  when 
two  or  more  tasks  are  conducted  concurrently  they  compete 
for  this  resource  (Keele,  1973'*^)*  Moreover  even  within  a 
single  task  the  teacher  should  proceed  from  simple  to  more 
complex  examples  of  each.  Gradually,  as  practice  proceeds 
the  individual  reaches  a  level  of  increased  automaticity 
of  responding  so  that  the  once  very  attention  demanding 
task's  drain  on  central  processing  capacity  is  reduced  and 
more  complex  behavior  may  be  efficiently  undertaken. 

With  the  distinction  of  open  and  closed  skills  esta¬ 
blished  along  with  the  importance  of  resource  allocations 
reiterated,  it  may  be  pointed  out  that  theoretical  develop¬ 
ments  in  the  learning  of  motor  skills  have  centered  around 
a  component  analysis  as  discussed  previously.  The  HPM 
breaks  the  composite  into  perceptual,  decision  and  effector 
components.  The  nature  of  these  components  differ  however 
between  open  and  closed  skills. 
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Open  Skills. 

Perceptual  Mechanism.  The  perceptual  mechanisms  code,  orga¬ 
nize,  and  transfer  the  incoming  information.  Such  informa¬ 
tion  is  used  by  both  the  decision-making  component  and  the 
memory  mechanisms  responsible  for  storing  information  for 
later  use.  The  perceptual  problems  for  the  performer  con¬ 
sist  of: 

(1)  anticipating  the  ways  in  which  the  stimuli  will 
change  and  monitoring  the  target  as  it  in  fact 
moves  through  time  and  space; 

(2)  at  the  same  time  the  performer  has  to  monitor  his 
own  performance  in  tracking  the  target  through 

an  analysis  of  his  own  response  behavior  and  an 
awareness  of  his  position  relative  to  the  target; 

(3)  the  performer  must  also  incorporate  the  extrinsic 
and  intrinsic  feedback  available  while  the  response 
is  being  executed  (performance  feedback),  and 
when  the  response  is  completed. 

In  an  open  skill  the  most  important  determiner  of  the  accu¬ 
racy  of  a  response,  then,  is  the  ability  to  predict  and 
anticipate  what  the  ’’opponent"  or  target  will  do. 

Once  achieved,  this  ability  will  allow  the  performer 
to  selectively  attend  to  only  a  subset  of  the  stimuli  on 
the  display,  thereby  reducing  the  load  on  central  processing 
capacity,  and  making  the  nsystemn  more  able  to  respond  to 
other  stimuli.  It  will  also  give  the  performer  the  added 
benefit  of  a  decreased  response  latency* 

Decision  Mechanism.  A  characteristic  of  an  open  skill  is 
that  maxy  plans  of  action  may  be  possible  for  any  given  input 
configuration.  The  goal  is  to  decide  on  an  appropriate 
plan  of  action  to  meet  optimally  the  need  of  the  situation. 
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The  efficient  performer  will  have  the  ability  to 
bring  the  most  likely  plans  of  action  or  schema  out  of 
long-term,  and  into  short-term  memory.  The  obvious  advan¬ 
tage  of  this  ability  is  the  reduction  of  alternative  plans 
of  action  that  the  decision  system  will  ultimately  have  to 
scrutinize  before  affecting  a  response,  and  thus  a  savings 
in  execution  time.  The  inexperienced  performer,  lacking 
either  the  storage  of,  or  the  ability  to  retrieve  the 
appropriate  subset  will  find  himself  stymied  by  the  complex 
input  which  by  definition  is  continually  changing  in  an 
open  skill  task.  This  is  essentially  the  pre-processing 
idea  discussed  earlier. 

The  major  source  of  vulnerability  to  the  efficiency 
of  the  system  vis  a  vis  the  decision  process,  is  of  course 
the  opponent's  behavior  changing  while  the  performer  is 
choosing,  or  having  changed  once  the  performer  has  chosen 
a  plan  of  action.  Ideally  the  performer  should  be  able  to 
make  a  fast  enough  analysis  of  the  situation  so  as  to  be 
able  to  nullify  or  successfully  react  to  the  opponent's 
behavior. 

Effector  Mechanism.  The  effector  mechanism  may  be  considered 
ft  1  organized  both  hierarchically  and  sequentially.  The 
01  r  implies  that  the  strategy  must  be  determined  before 
the  tactics  can  be  initiated.  The  sequencing  of  informa¬ 
tion  is  also  important  since  once  the  components  of  the 
plan  of  action  are  determined,  their  execution  order  must 
also  be  specified  to  meet  the  objectives  of  the  task. 

...  A  major  difference  between  the  organi¬ 
zation  of  the  effector  mechanism  of  an  exper¬ 
ienced  (performer)  and  a  novice  is  that  the 
former  has  a  number  of  "motor  programs"  at 
his  command.  Motor  programs,  from  this  point 
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of  view  would  be  highly  overleamed  plans 
of  action  stored  somewhere  in  the  brain 
and  capable  of  being  run  off  automatically 
once  the  performer  has  ordered  their  execu¬ 
tion.  .  1976,  page  26110) 

1  Note  that  the  notion  of  motor  programs  is  not  universally 
accepted  (see  Adams,  1971 yet  it  has  sufficient 
"functional  utility"  to  be  included  here.  ] 

Closed  Skills.  There  are  two  basic  sources  of  distinction 
between  open  and  closed  skills.  In  the  Marteniuk  presenta¬ 
tion  of  the  HPM  it  is  assumed  that  time  stress  generated 
by  complex  perceptual  and  decision  processes  does  not 
typically  exist  for  closed  skills.  This  is  certainly  the 
case  for  the  athletic  skill  of,  say,  a  golf  shot  were  the 
player  may  take  virtually  all  the  time  necessary  to  prepare 
for  the  response.  On  the  other  hand,  closed  skills  such 
as  responding  in  the  constant  and  predetermined  manner  to 
a  given  visual  code  may  very  well  be  subject  to  time 
pressures.  It  should  be  recognized  then  that  the  following 
HPM  model  may  be  limited  in  its  application  to  time-stressed 
closed  skills. 

The  second  major  distinction  is  in  the  demands  placed 
upon  the  decision  and  effector  components.  A  closed  skill 
requires  the  execution  of  a  specific  plan  of  action  when  the 
performer  is  confronted  with  a  given  stimulus  configuration. 
The  flexibility  and  diversification  required  of  the  system 
in  an  open  skill  are  less  necessary  for  the  efficient  per¬ 
formance  of  a  closed-skill  task. 


Perceptual  Mechanisms.  While  the  above  statements  concern¬ 
ing  flexibility,  etc.,  are  true,  it  is  nonetheless  the  case 
that  in  a  closed  skill  the  performer  must  still  process  the 
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various  sources  of  information  before  executing  the  response, 
and  must  still  attend  to  the  feedback  during  performance 
as  well  as  upon  completion  of  the  response.  The  same  general 
requirements  of  the  perceptual  system  apply  for  closed  as 
for  open  skills. 

Effector  Mechanism,  In  a  closed  skill  the  notion  of  a  motor 
program  becomes  even  more  intuitively  (if  not  empirically) 
valid.  It  is  to  be  considered  that  once  the  incoming 
information  has  been  analyzed  and  classified,  the  degrees 
of  freedom  for  response  selection  have  virtually  been  reduced 
to  zero.  Only  one  correct  course  of  action  is  appropriate 
and  if  contained  in  the  memory  system  of  the  individual, 

ry  c 

should  occur  without  hesitation.  In  Adams  (1971  )' 
terminology,  the  "memory  trace”  should,  by  a  simple  associa¬ 
tive  process,  elicit  the  oorrect  movement  sequence. 

However,  one  may  hesitate  to  characterize  this  process 
as  being  purely  the  running  off  of  a  simple  motor  program. 
There  are  sufficient  examples  of  individuals  well  skilled 
in  the  execution  of  complex  sequential  behavior  (e.g. 
professional  violinists,  and  typists)  who  in  the  midst  of 
apparently  being  under  the  influence  of  a  well  defined 
motor  program,  still  have  the  admirable  ability  to  monitor 
their  own  behavior  and  to  detect  errors. 

Communication.  In  using  the  HPM  to  develop  principles  for 
teaching  some  general  considerations  must  be  noted.  In 
teaching, 


•  ,  .  the  main  concern  is  with  the  naive 
or  relatively  inexperienced  performer 
who  has  not  accumulated  this  huge  reser¬ 
voir  of  past  experience  to  draw  upon. 
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What  this  means  is  that  individuals  of 
this  type  are  faced  with  a  great  deal 
more  information  (that  is,  uncertainty) 
than  someone  with  experience.  The 
naive  performer  has  no  basis  for  making 
absolute  Judgments  or  anticipating  en¬ 
vironmental  events.  At  the  same  time 
he  has  not  yet  established  compatability 
between  various  environmental  demands  and 
their  proper  actions  and  further,  has  had 
very  little  practice  in  organizing  and 
executing  plans  of  action. 

(Marteniuk,  1976,  page  29^^) 


What  the  above  essentially  gets  reduced  to  is  a  subject 
potentially  confronted  with  huge  amounts  of  information 
and  not  having  the  wherewithal  to  respond  to  them  The 
teaching  situation  must  be  structured  to  limit  the  amount 
of  information  during  the  acquisition  procedures.  One 
obvious  way  to  proceed  with  acquisition,  then,  would  be  to 
slow  down  the  rate  of  presentation  of  information  to 
greater  than  real-time  intervals.  This  has  the  advantage 
of  taking  the  learner  slowly  through  the  steps  in  the 
response  sequence  as  well  as  familiarizing  the  learner 
with  the  characteristics  of  the  sources  of  feedback 
available  during  and  after  responding. 


A  second  obvious  procedure  would  be  to  limit  the 
complexity  of  the  tasks  so  as  to  present  them  in  subsets 
of  related  activities  until  some  criterion  is  reached 
before  moving  on  to  the  more  complex  integration  of  these 
tasks . 


Perceptual  Mechanisms.  The  first  consideration  is  that 
the  subject  be  motivate^*  The  limited  short-term  memory 
functions  dictate  that  new  information  be  presented  in 
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such  a  way  as  to  allow  the  learner  to  remember  over  the 
learning  trials,  relevant  information  about  the  task.  The 
instructions  too  must  not  exceed  the  limited  ability  of  the 
subject  to  remember  them  in  initial  learning  trials.  Ampli¬ 
fication  and  expansion  should  come  only  after  preliminary 
routines  have  been  mastered  (i.e.  put  into  long-term  storage). 
Of  consideration  here  also  is  the  degree  to  which  a  complex 
task  should  be  subdivided  for  basic  mastery.  Based  upon  the 
Fitts  and  Posner's  (1976)*^  review,  one  would,  when  con¬ 
fronted  with  the  necessity  to  subdivide  a  task  based  upon 
initial  perceptual  and  memorial  limitations,  arrange  the 
components  so  that  they  are  as  independent  of  each  other  as 
possible.  Components  that  will  ultimately  require  a  high 
degree  of  integration  and  interdependence  should  be  prac¬ 
ticed  together, >et  be  separated  from  components  from  which 
they  are  relatively  independent. 


With  respect  to  content,  the  teacher  must,  especially 
with  open  skills,  ins time t  or  prepare  the  student  as  to 
what  dimensions  or  input  characteristics  to  respond  to. 

Thus  the  teacher  primes  the  learner  to  anticipate  the 
relevant  changes  in  the  input  and  early  in  training  helps 
the  learner  to  limit  the  set  response  alternatives  relevant 
to  a  given  situation. 

Decision  Mechanism.  In  open  3kills  the  teacher  should 
make  clear  what  the  relationships  are  between  the  environ¬ 
ment  and  alternative  responses.  This  becomes  more  impor¬ 
tant  the  more  complex  the  input  situation  becomes. 

Effector  Mechanism.  Up  to  this  point  little  has  been  said 
concerning  the  cognitive  component  of  motor  skills.  Adams 
(1971)^'*  has  stated  that  it  is  difficult  to  consider  a 
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motor  skill  being  acquired  with  the  total  absence  of  verbal 
control  during  the  initial  stages.  The  ubiquitous  verbal 
mediation  and  organization  that  humans  use,  especially 
during  early  learning  trials ,  may  be  made  more  efficient 
if  prepared  in. advance  by  the  teacher  and  presented  to  the 
learner.  Thus,  in  executing  a  complex  sequence  of  behavior 
it  would  be  helpful  to  have  a  verbal  representation  of  the 
sequencing  of  the  responses  to  organize  the  total,  more 
complex,  response.  Eventually  this  verbal  mediator  will 
drop  out  but  nonetheless  will  be  retained  in  long-term 
storage  should  the  execution  pattern  for  some  reason  break 
down.  In  other  words  having  the  verbal  sequencing  will  ex¬ 
pedite  mastery  of  the  task,  and  may  even  be  a  valuable  re¬ 
source  in  an  emergency  situation  or  high  stress  situation 
when  the  automaticity  of  responding  breaks  down. 

The  pedogogical  recommendations  resulting  from  the 
HPM  implies  any  means  by  which  the  teacher  can  limit 
uncertainty  in  decision-making  will  facilitate  performance 
in  the  task.  Early  in  training  this  is  particularly  impor¬ 
tant  since  it  reduces  processing  capacity  utilization  for 
decision  mechanisms  and  places  emphasis  on  the  execution 
of  a  proper  response.  The  goals  of  the  behavior  should  be 
made  perfectly  clear  to  the  student.  This  is  especially 
important  for  complex  behaviors  where  the  component  res¬ 
ponses  'relationships  with  the  overall  response  should  be 
made  clear.  Also  controlled  should  be  students'  expecta¬ 
tions  of  potential  changes  in  the  stimulus  array.  This 
will  ensure  that  the  student  pays  attention  to  relevant 
dimensions  on  the  display.  Moreover,  especially  early  in 
practice,  the  number  of  events  to  be  attended  to  should  be 
restricted  to  allow  the  student  the  opportunity  to  master 
simpler  components  before  moving  on. 
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The  initial  steps  towards  aohleving  automaticity  may 
begin  with  the  instructor  providing  the  student  with  a 
verbal  outline  of  the  plan  of  action  for  the  desired  move¬ 
ment*  This  mnemonic  device  will  allow  the  student  to 
store  in  memory  a  representation  of  the  movement  and  its 
sequence  that  he,  early  in  practice,  could  not  otherwise 
be  expected  to  have.  The  mnemonic  will  in  particular 
provide  a  verbal  step-by-step  plan  of  action  that  will 
assure  the  proper  sequencing  of  the  more  complex  movements. 
Ultimately  this  mnemonic  may  not  be  needed  as  automaticity 
is  achieved  but  as  stated  earlier,  it  may  at  sometime  be 
a  useful  long-term  memory  resource. 

The  use  of  all  forms  of  feedback  is  very  important 
for  the  development  of  efficient  motor  schema.  The 
following  recommendations  are  made  to  optimize  the  feed¬ 
back  made  available  to  the  student.  The  feedback  must  be 
presented  in  a  form  that  the  student  will  be  able  to 
utilize.  The  task  should  be  augmented  so  that  the  student, 
after  presentation  of  feedback,  has  in  short- term  memory 
the  goal  of  the  movement,  the  knowledge  of  the  outcome  of 
the  movement,  the  image  and  plan  for  action  of  the  move¬ 
ment,  and  the  actual  way  the  movement  was  executed.  These 

are  essentially  the  components  of  a  motor  response  that 
76 

Schmidt  (1975)'  has  defined  as  the  necessary  information 
needed  to  develop  a  motor  schema,  or  rules  for  a  parti¬ 
cular  movement.  A  final  important  point  here  is  that  the 
student  should  have  some  uninterrupted  time  after  response 
execution  to  evaluate  for  himself  the  response  just 
completed  and  the  feedback  received. 
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Instructional  Strategies.  It  has  been  previously  stated 
that  skills  research  has  lagged  behind  other  areas  of  . 
scientific  endeavor  in  the  area  of  learning  and  memory, 
and  that  only  relatively  recently  have  unifying  theore¬ 
tical  concepts  been  attempted.  Though  the  currently  most 
popular  theory  (Schmidt,  1975'  )  is  now  several  years  old, 
it  has  had  relatively  little  impact  on  applied  research. 
Fortunately  for  the  issue  at  hand  in  this  report,  there 
has  been  a  series  of  experiments  conducted  by  Singer  and 
his  associates  (Singer,  1975] 12  Singer  and  Gaines,  1975; ^ 
Singer  and  Pease,  197611^)  and  by  Prather  and  his  co¬ 
workers  (Prather,  197l]^  Prather  and  Berry,  1970;^ 

117 

Prather,  Berry  and  Bermudez,  1972  )  that  seem  to  relate 

to  Schmidt's  thinking. 

Singer's  research  especially  has  been  concerned  with 
the  assessment  of  instructional  strategies  and  their  effects 
on  the  acquisition,  retention,  and  transfer  of  a  complex 
motor  response.  His  basic  apparatus  is  sufficiently  simi¬ 
lar  to  the  AIC  console  to  warrant  more  than  just  a  casual 
review  here  (see  Singer  and  Pease,  197611**  Tor  a  complete 
description.)  The  Singer  and  Pease  study  (1976)^^ 
serves  as  a  general  reference  for  their  research  and  will 
be  reviewed  here  in  some  detail.  The  task  involved  a 
computer-managed  serial  manipulation  apparatus  which  re¬ 
quired  sequential  coordinated  hand  and  foot  movements. 

The  eight  common  objects  (door  handle,  pound 
buttons,  and  various  types  of  switches) 
yielded  a  total  possibility  of  19  different 
manipulations  from  which  the  learner  could 
choose  when  making  a  selection  for  the  first 
correct  response.  Four  of  the  hand  objects 
were  wired  to  a  foot  pedal  in  such  a  way 
that  they  could  operate  alone  or  in  a  series 
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with  the  foot  pedal.  The  subject  had  to 
press  the  foot  pedal  simultaneously  with 
the  hand  manipulation.  Prom  the  19  possi¬ 
ble  manipulations ,  8  were  selected  to  be 
programmed  as  a  serial  manipulation 
sequence  for  the  initial  learning  program. 

(Singer  and  Pease,  1976,  page  791^^) 

Feedback  was  presented  by  the  visual  display  of  lights 
and  indicated  correctness  of  response.  The  sequence  was 
changed  on  a  subsequent  transfer  task.  Of  central  concern 
were  the  effects  on  learning  of  three  different  types  of 
learning  strategies,  with  male  and  female  undergraduate 
subjects  aged  17  to  25  years  being  randomly  assigned  to 
treatment  conditions . 

All  subjects  studied  a  written  description  of  the 
sequence  two  minutes  before  learning  commenced.  The"guided" 
learning  group  received  prompting  in  the  form  of  a  visual 
display  of  a  number  corresponding  to  the  next  object  to  be 
manipulated,  as  well  as  verbal  cues  (not  defined  in  the 
study)  by  the  experimenter.  The  "discovery”  or  what  Prather 
has  called  in  his  research,  the  "trial  and  error"  group, 
received  neither  cues  nor  prompts.  Finally,  a  combination 
group  received  prompting  and  cueing  for  the  first  four 
trials  but  nothing  for  the  remaining  16  trials  of  acqui¬ 
sition.  All  groups  performed  20  trials  of  the  task. 

The  next  day  subjects  returned  for  a  retention  test 
on  the  initial  sequence  with  no  cuing  or  prompting  for 
any  group.  There  then  followed  a  short  relearning  phase 
to  establish  uniformity  of  responding,  followed  by  a 
transfer  task  of  16  trials  administered  to  all  subjects 
using  the  "discovery”  method. 
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The  following  results  were  obtained.  Subjects  on  the 
"guided"  task  mastered  the  initial  task  faster  than  any 
Other  group.  However*  the  two  groups  with  some  discovery 
experience  evidenced  superior  and  equal  retention  to  the 
guided  group,  and  superiority  to  the  guided  group  in  earlier 
stages  of  the  transfer  task.  The  authors  stress  that  the 
inclusion  of  a  "discovery  component"  will  create  a  more 
flexible  motoric  representation  that  will  benefit  the  sub¬ 
ject  not  only  on  the  specific  practice  task  but  on  related 
ones  as  well. 


The  implication  of  these  data  for  open  skill  tasks  is 
that  subjects  should  be  presented  with  a  variety  of  stimulus 
situations  and  have  the  opportunity  to  respond  in  a  variety 
of  ways  at  some  point  during  practice.  Though  Singer  and 
Pease  introduce  discovery  later  in  training  for  the  mixed 
condition,  the  effects  of  an  initial  discovery  period  fol-  ^ 

lowed  by  guided  learning  remains  to  be  determined.  Thus, 
the  optimum  sequencing  of  discovery  and  guided  learning 
has  yet  to  be  decided. 

These  data  fit  nicely  with  the  requirement  that  open 
skill  tasks  be  learned  with  the  subject  being  aware  of  dif¬ 
ferent  response  possibilities  (plans  of  action)  and  is  in 
agreement  with  the  conclusion  of  the  several  researchers 
(mentioned  earlier)  who  have  investigated  Schmidt's  theory 
and  who  have  found  superior  learning  and  transfer  with 
variability  during  acquisition. 
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TOWARD  A  THEORY  OP  THE  EXPERT  AND  NOVICE 

If  we  could  aggregate  the  Important  features  of  the 
three  controller  tasks  (GCA,  LSO,  and  AIC)  into  a  single 
hypothetical  controller’s  role,  we  may  find  that  the  dif¬ 
ferences  between  the  expert  and  novice  can  be  described 
quite  well  in  light  of  the  recent  basic  research  and  theory 
just  discussed.  Consider,  if  you  will,  this  composite 
controller,  seated  at  a  console  monitoring  a  display  which 
presents  a  continuous  flow  of  visual  input  with  inter¬ 
mittent  auditory  input  over  a  headset.  The  visual  events 
are  predictable  with  experience,  but  some  of  the  visual 
cues  can  be  quite  subtle  and  will  require  much  practice  in 
order  to  perceive  them.  Information  load  and  processing 
demands  will  be  quite  heavy  at  times  as  there  may  be  simul¬ 
taneous  visual  inputs — not  to  mention  auditory  inputs— 
concurrent  with  verbal  and  motor  response  requirements. 

Some  of  the  responses  may  be  associated  with  a  single 
visual  input  but  most  of  the  responses  are  to  be  elicited 
by  a  complex  combination  of  events.  The  responses  themselves 
vary  from  simple  to  complex  constructed  verbal  responses 
and  sequenced  motor  responses. 

Expert 

Perceptual.  In  describing  the  expert’s  perceptual  skills, 
we  could  describe  two  possible  situations*  The  first  is 
that  the  perceptual  schemata  by  which  the  subtle  cues  are 
picked  up  are  unitized.  Through  extensive  practice,  the 
expert  automatically  codes  the  features.  The  second  possi¬ 
bility  is  that  the  predictability  of  the  visual  inputs 
leads  the  expert  to  activate  schemata  in  anticipation 
resulting  in  detections  with  very  short  reaction  times. 

( 
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Needless  to  say,  both  the  automatlclty  and  expectation 
explanations  may  be  correct  to  some  degree.  (See  page  58). 

Visual  Search.  Because  of  the  rapidity  with  which  the  visual 
inputs  are  coded,  as  described  above,  additional  simultan¬ 
eously  presented  inputs  do  not  appreciably  increase  pro¬ 
cessing  load  as  though  they  were  processed  in  parallel.  In 
the  case  of  divided  displays,  expectations  lead  to  effi¬ 
cient  scanning  responses . 

Memory  Search.  The  inputs  are  compared  with  a  number  of 
events  for  which  the  expert  is  looking.  The  comparison 
process  has  become  automatic  to  the  point  that  the  number 
of  events  for  which  he  is  searching  no  longer  effects 
search  rates. 

Response  Selection  and  Construction.  First,  simple  struc¬ 
tures  are  unitized  such  that  a  specific  visual  input  auto¬ 
matically  elicits  the  appropriate  response.  Secondly, 
those  response  selections  dependent  upon  a  complex  combin¬ 
ation  of  inputs  may  be  helped  by  pre-processing.  As  an 
example  of  this  pre-processing,  look  at  Figure  16  wherein 
the  expert  is  to  compute  a  final  heading  to  be  given  to  the 
pilot.  Instead  of  waiting  until  the  heading  is  needed, 
he  anticipates  its  need  by  processing  information  during 
a  preceding  advisory.  Thus  while  in  the  act  of  giving  an 
elevation  advisory  for  example,  he  notes  that  the  last 
heading  given  was  160°,  the  aircraft  is  within  five  miles 
of  touchdown,  and  there  is  a  crosswind  with  which  he  must 
reckon.  Noting  all  of  this,  he  pre-selects  two  possible 
headings.  If  the  pilot  is  late  and  will  overshoot  the 
azimuth,  a  correction  of  164°  is  to  be  given— otherwise 
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Figure  16.  Schematic  Diagram  of  an  Example 
of  Preprocessing 

( 
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162°  will  be  given.  These  two  headings  may  be  held  in 
short-term  memory  until  the  requisite  visual  input  is 
received  by  which  the  controller  will  select  and  execute 
one  of  the  two  advisories.  Without  the  preprocessing, 
when  the  requisite  visual  input  is  received,  the  controller 
would  have  to  begin  at  the  top  of  the  diagram  in  Figure  16 
and  must  invest  resources  and  time  between  the  onset  of  the 
visual  input  and  the  output  of  the  response. 

This  form  of  preprocessing  (similar  to  that  reported 
by  our  expert  PAR  controller)  is  a  likely  description  for 
the  expectation  mechanism.  Unanticipated  events  (e.g.  the 
pilot  reacting  early  instead  of  late)  would  require  a  heading 
response  not  held  in  short-term  memory.  Thus  the  expert 
would  either  give  an  error  or  have  to  go  through  time  con¬ 
suming  reprocessing  to  select  a  response. 

Thirdly  internal  schemata  have  been  modified  over  the 
lengthy  time  the  expert  has  been  functioning  in  his  capa¬ 
city.  Desriptions  are  so  finely  tuned  that  mismatches 
rarely  occur.  Parts  of  the  knowledge  structure  have  been 
reformulated  such  that  needless  steps  in  processing  have 
been  eliminated  so  that  now  the  resource  allocation  func¬ 
tions  look  like  "D"  in  Figure  8. 

Response  Execution.  For  closed-skill  tasks,  the  expert  has 
well  established  "motor  programs"  upon  which  to  draw.  He 
can  predict  the  sequential  steps  in  the  overall  monitoring 
situation  and  is  waiting  only  for  the  necessary  information 
to  trigger  the  particular  program  sequence.  The  task  has 
been  mastered  to  such  a  level  that  little  in  the  way  of 
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resource  allocation  is  needed.  Most  of  whatever  resources 
are  required  in  motor  responding  are  given  over  to  the 
open-skill  tasks  which  require  constant  monitoring. 

For  the  open-skill  task  of  tracking  a  target  on  a  con¬ 
tinuously  changing  display,  the  expert  has  Tad  enough  exper¬ 
ience  to  predict  certain  general  types  of  target  changes. 

The  expert  will  vary  the  amount  of  processing  capacity 
given  to  the  task  as  a  function  of  the  total  current  state 
of  the  operation  and  the  importance  of  the  tracking  task 
on  the  overall  execution  of  the  task.  As  the  situation 
becomes  more  crucial,  the  expert  must  transfer  more  pro¬ 
cessing  capacity  to  the  tracking  task,  and  must  retrieve  and 
keep  more  of  the  alternative  plans  of  action  in  short-term 
memory  to  keep  the  system  flexible. 

Thus  the  performance  of  the  expert,  for  the  most  part, 
can  be  said  to  be  in  the  data-limited  region.  What  auto¬ 
matic  processing  exists  makes  him  quite  rigid  in  terms  of 
any  reversals  or  modifications  that  might  be  required  in 
retraining.  His  automatic  detection  skills  also  make  him 
highly  susceptible  to  distraction  if  told  for  some  reason 
to  ignore  certain  inputs.  Because  of  his  reliance  on 
expectancies,  unexpected  events  will  throw  him — requiring 
that  he  process,  displaying  increased  investment  of  effort. 
Finally,  because  he  is  now  a  resource-efficient  system,  he 
does  not  display  problems  of  holding  certain  information, 
such  as  the  last  heading  given  in  short-term  memory.  He 
has  plenty  of  time  and  resources  left  for  the  recycling  of 
temporary  information. 


Novice 

Perceptual.  In  his  novice  state,  he  is  now  said  to  be  in 
either  the  feature  discovery  stage  or  the  coding  stage  of 
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perceptual  learning.  Here,  the  act  of  learning  itself, 
e.g.  the  search,  discovery  and  testing  of  features,  re¬ 
quires  an  investment  of  resources,  additional  to  that  re¬ 
quired  in  the  task.  Thus  the  resource  allocation  acquisi¬ 
tion  function  would  be  similar  to  that  shown  in  Figure  11. 

At  the  beginning  stages  of  perceptual  learning,  the  process 
may  be  data-limited  in  that  regardless  of  how  hard  he  tries, 
the  novice  LSO  for  example  may  not  be  able  to  estimate  the 
pitch  of  the  aircraft  because  he  had  not  identified  the 
relevant  features.  Later  the  process  becomes  resource- 
limited  in  that  (in  knowing  the  relevant  features)  he  must 
invest  resources  in  scanning  the  visual  array  of  features 
in  order  to  code  it.  Finally,  the  process  becomes  data- 
limited  again  as  the  coding  becomes  automatic  and  requires 
little  resources. 

Visual  Search.  The  search  of  the  visual  inputs  is  a  con¬ 
trolled  search  for  the  novice  and  requires  extensive 
resources.  With  the  display  divided,  he  anticipates  little, 
showing  inefficiency  in  his  scanning  behavior. 

Memory  Search.  Again,  the  student  is  engaged  in  controlled 
processing  instead  of  automatic  processing.  The  search 
itself  consumes  time  and  resources.  Targets  are  missed 
because  of: 

(1)  time  constraints  (the  amount  of  time  available 
for  detection  is  shorter  than  that  required  by 
controlled  processing  at  a  given  level  of  resource 
investment, 

(2)  resource  limitations  (because  of  competing  con¬ 
current  tasks  with  perhaps  higher  subjective 
utility  values,  the  amount  of  resources  available 
for  investment  is  deficient), 
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(3)  mismatch  as  between  target  and  memory  representa¬ 
tion  (primitive  schemata,  not  yet  finely  tuned, 
may  not  be  general  enough  for  detection  of  variable 
real-world  targets). 

Response  Selection.  Not  having  the  experience  to  anticipate 
events,  the  novice  seems  to  do  little  preprocessing.  Using 
the  example  shown  in  Figure  16,  instead  of  pre— computing  the 
two  possible  headings  of  164°  and  162°  in  anticipation  of 
the  need,  he  only  computes  the  required  heading  as  the  need 
presents  itself.  Thus  there  is  a  long  latency  between  the 
onset  of  the  need  for  the  response,  and  the  actual  produc¬ 
tion  of  the  response.  If  there  is  too  little  time  for  this 
multistep  processing,  either  the  response  will  be  omitted, 
or  certain  steps  will  be  omitted  with  default  values 
assumed  (e.g.  assume  no  wind  and  make  no  reference  to  the 
five  mile  hashmark  defaulting  to  the  five-degree  minimum 
course  change  rule)  resulting  in  an  error  response  (e.g. 
his  choice  might  be  between  165°  and  160°). 

The  novice  has  only  rudimentary  motor  programs  for  the 
execution  of  closed-skill  tasks.  Responses  that  the  expert 
can  make  almost  unknowingly  (without  resource),  the  novice 
must  not  only  take  time  to  retrieve  and  decide  upon,  but 
most  likely  take  time  to  execute  as  well.  The  obvious 
drain  on  resources  detracts  from  those  components  of  the 
task  requiring  larger  investments,  keeps  the  novice  from 
"keeping  up"  with  the  real-time  situation,  and  in  an  over¬ 
all  sense,  decreases  the  efficiency  of  the  system. 

The  novice  does  net  have  the  developed  memory  ability 
to  preselect  response  patterns  for  a  recognized  subset  of 
alternative  target  behaviors.  The  novice  maintains  an 
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awkwardly  large  memory  set,  often  of  actvially  irrelevant 
material.  Unlike  the  expert  who  can,  with  confidence, 
limit  the  size  of  the  set  of  alternative  plans  of  action, 
the  novice  gingerly  holds  on  to  plans  of  action  even  though 
the  expert  would  long  before  have  deemed  them  as  "unlikely 
occurences,"  and  would  have  precluded  them  from  being  a 
drain  on  resources. 

Thus  the  novice,  for  the  most  part,  functions  in  a 
resource-limited  domain.  He  is  characterized  by  controlled 
rather  than  automatic  processing.  He  handles  high  frequency 
events  as  well  as  rare  events,  because  he  has  not  built  up 
any  expectancies  and  he  does  not  engage  in  much  preprocessing 
Because  he  is  in  controlled  processing  and  not  automatic,  he 
is  quite  flexible  regarding  reversals  of  contingencies  and 
training  modifications  in  general.  Since  he  is  characterized 
by  time  and  resource  consuming  controlled  processing,  he 
becomes  overloaded  easily  and  may  lose  information  from 
short-term  memory  because  his  resources  were  diverted  from 
any  recycling  process. 

Having  described  the  expert  and  novice  in  terms  of 
current  research  in  cognition,  the  next  task  becomes  one 
of  reviewing  current  developments  in  training.  The  next 
section  reviews  recent  developments  in  CAX,  primarily 
regarding  the  representation  of  knowledge  and  tutorial 
strategies.  Most  of  the  developments  have  not  dealt  with 
time-sharing  tasks  (a  point  we  will  want  to  address  later), 
but  show  promise  in  their  particular  approach  taken, 
particularly  the  utilization  of  artificial  intelligence  in 
CAI. 
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DEVELOPMENTS  IN  CAI 

Computer  assisted  instruction  (CAl)  has  come  a  long 
way  since  the  early  1960s.  Its  development  has  been  influ¬ 
enced,  by  basic  research  in  cognition,  artificial  intelli¬ 
gence,  and  linguistics.  The  recent  history  of  CAI  has 
been  described  by  Lacey  (1977)  McLagan  and  Sandborgh 
(I977)^^and  Suppes  and  Macken  (1978)1^  Feurzeig,  Cohen, 
Lukas  and  Schiff  (1975)  provide  a  review  on  research 
development  in  adaptive  training.  Early  programs  in  CAI 
were  developed  at  the  Institute  of  Mathematical  Studies  in 
the  Social  Sciences  (IMSSS)  at  Stanford  (see  Atkinson, 

1p68j  Suppes  and  Mornings tar,  1972  J)  the  University 
of  Texas  at  Austin,  (see  Judd,  Bunderson,  and  Bessent, 
197012^),  Florida  State  University  (see  Hagerty,  1970^2"*) 
and  of  course  the  University  of  Illinois  wherein  the  well- 
known  PLATO  system  was  developed  in  connection  with  the 
National  Science  Foundation  and  the  Control  Data  Corporation. 
The  initial  results  of  CAI  met  with  mixed  results  when  com¬ 
pared  with  traditional  instruction  (see  Vinsonhaler  and 
1  26 

Bass,  1972  ).  Early  1970s  brought  widespread  use  of 

authoring  languages  such  as  Courpewriter  developed  by  IBM 
back  in  the  early  60s,  the  use  of  minicomputers  and  tele¬ 
vision  technology  such  as  the  TICCIT  system  developed  by 
the  MITRE  Corporation,  and  the  introduction  of  artificial 
intelligence  in  CAI  by  Bolt^  Beranek  and  Newman,  Inc. 

Recent  trends  have  been  toward  miniaturization,  the  devel¬ 
opment  of  intelligent,  knowledge-based  systems  with  natural 
language  capabilities,  and  speech  recognition/production 
(see  Atkinson,  1978  '  regarding  CAI  in  the  future). 
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Of*  all  the  developments  in  CAI,  those  of  greatest 
possible  impact  on  future  automated-speech-based  training 
systems  would  be:  (l)  knowledge  representations ,  (2)  optimi¬ 
zation  techniques,  and  (3)  natural  language  capabilities. 

(it  goes  without  saying  that  the  speech  recognition  tech¬ 
nology  itself  would  be  of  utmost  importance.)  The  sections 
that  follow  will  attempt  to  describe  developments  in  the 
representation  of  knowledge  and  natural  language  processing. 
Chatfield  and  Gidcumb  (1977)"^  provide  a  review  of 
optimization  techniques  which  will  not  be  reviewed  here. 


REPRESENTATION  OF  KNOWLEDGE.  The  goal  of  research  efforts 
in  CAI,  even  back  in  the  60*s  was  to  develop  systems  which 
would  give  the  student  some  amount  of  initiative  in  the 
process.  Bryan  (1969)  developed  a  taxonomy  by  which  he 
categorized  the  developments  in  CAI  in  terms  of  the  amount 
of  initiative  the  student  could  take  with  the  system.  The 
first  category  in  which  the  student  was  allowed  the  most 
freedom  was  referred  to  as  ad  lib  CAI.  Here  the  student  is 
given  full  control  of  a  library  of  routines  which  he  uses 
at  will.  An  example  of  this  was  LOGO  developed  by 
Feurzeig  and  Papert  (1970)»  y  The  second  category  consisted 
of  games  and  simulation  wherein  the  student  was  allowed  an 
intermediate  amount  of  initiative.  An  example  of  this  was 
the  Socratic  system  by  Swets  and  Feurzeig  (1965). J  In  the 
last  category  the  student  was  maximally  constrained  by 
simply  being  given  a  preconceived  series  of  frames  with 
Crawderian  branching  as  the  only  novelties  in  student 
trajectories  through  the  curriculum.  These  deterministic 
frame-based  systems  have  been  variously  referred  to  as 
script-based  CAI  and  ad  hoc  frame-oriented  (HFO)  systems. 
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Since  Bryan's  review,  developments  in  CAI  have  pro¬ 
duced  mixed  initiative,  generative  systems  employing  tech¬ 
niques  from  artificial  intelligence  (Al).  The  research  by 
Jaime  Carbonell  (1970)*  funded  by  ONR,  could  be  said  to 
be  the  turning  point  in  this  direction.  Carbonell  sought 
a  mixed- initiative  discourse  between  the  student  and  the 
system.  It  was  easy  enough  for  the  script-based  systems  to 
query  the  student,  but  not  vice-versa.  To  enable  the  student 
to  query  the  system  two  requirements  must  be  met: 

(1)  the  system  must  be  able  to  understand  semanti¬ 
cally  as  well  as  syntactically,  novel  natural- 
language-  type  inputs  on  the  part  of  the  student, 

(2)  the  system  must  somehow  have  incorporated  the 
knowledge  of  the  expert  so  that  it  can  "generate" 
responses . 

At  the  time,  the  products  of  basic  research  efforts  in  AI 
(especially  from  the  MIT  Lab)  had  begun  to  Impact  both 
areas.  Bobrow  (1964)]*^  Simmons  ( 1 970a )]  "^  ( 1 970b )^"^  and 
Simmons  and  Silberman  (1967)  ^  had  carried  out  research  on 
natural  language  communication  with  computers,  while  Minsky 
(1968)  J  and  others  mentioned  previously  had  worked  on  the 
problem  of  representing  knowledge  in  the  computer. 

131 

Carbonell  (1970)  developed  a  training  system  called 
SCHOLAR  which  was  used  to  teach  geography.  Instead  of  the 
script  based  frame  oriented  methodology,  Carbonell  developed 
a  semantic  network  of  declarative  facts  comprising  the  know¬ 
ledge  domain.  A  schematic  drawing  of  a  portion  of  the  net¬ 
work  would  be  represented  by  Figure  17.  Associated  with 
each  concept  is  a  list  of  descriptors.  The  descriptors 
include  such  things  as  the:  super  concept,  examples  of  the 
concepts,  locations  or  other  detail  of  specification,  etc. 
Note  the  linkages  between  concepts.  It  is  possible  that 
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ilia 


Super  Concept — Country 
Location-— South  America  • 
Border  Countries— -Brazil 


South  America 


Continent 

Super  C— Body  of  Land  Surrounded 

by  Water 

Ins  tances  -  -North  America 

South  America  ■ 

Africa 

etc. 


C— Argentina 


Super  C — Country 
Location— South  America 
Border  Countries— Argentina 
etc. 


Figure  17-.  Example  Knowledge  Structures  in  the 
SCHOLAR  System 


some  of  the  items  listed  under  a  concept  may  themselves  be 
concepts  in  the  knowledge  domain.  Thus  the  information 
which  was  to  be  taught  was  represented  as  a  semantic  net¬ 
work  similar  to  proposals  of  the  representation  of  know¬ 
ledge  in  humans  as  described  previously.  Using  this  infor¬ 
mation,  stored  as  a  semantic  network  and  its  natural  langu¬ 
age  capabilities,  SCHOLAR  would  generate  questions  and 
answers  for  the  student.  An  example  dialogue  between 
SCHOLAR  and  the  student  is  given  in  Figure  18  and  is  taken 
from  Carbonell  (1970,  p  192-193) •  Phrases  printed  in  upper 
case  are  from  SCHOLAR,  with  the  student's  responses  in  lower 
case  and  an  asterisk  to  the  left. 
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THE  IANGUAGE  IN  ERAZIL  IS  PORTUGUESE. 

IS  IT  TRUE  THAT  THE  CAPITOL  IN  BRAZIL  IS  BRAZILIA? 


*v es. 

THAT'S  FINE. 

APPROXIMATELY  WHAT  IS  THE  POPULATION  IN  ERAZIL? 
*60000000. 


APPROXIMATELY  TRUE,  BUT  A  TORE  EXACT  VALUE  IS  70000000. 
WHAT  IS  BtAZIL? 


*tell  me  scene  thine  about  Peru. 

WAIT  A  MINUTE?I,M  CHECKING  MY  THIS. 
LET'S  PROCEED  NOW 

PERU  IS  A  COUNTRY. 

IT  IS  IN  SOUTH  AMERICA. 


THE  CAPITOL  IS  LIMA. 

NOW  ANSWER  THE  QUESTION  YOU  DIDN’T  ANSWER  BEFORE. 

*q/a. 

DO  YOU  WANT  TO  CHANGE  THE  GENERAL  MODE  OF  THE  INTERACTION 
TO  Q/A?  YES  OR  NO? 

*yes.  ' 

O.K.  I®.  JONES.  LET'S  GO  AHEAD. 

★please  tell  me  more  about  Peru. 

THE  AREA  OF  PERU  IS  APPROXIMATELY  480000  SQUARE 
MILES. 

THE  LANGUAGE  IS  SPANISH. 

★how  many  square  miles  in  Brazil? 

THE  AREA  IS  APPROXIMATELY  360000  SQUARE  MILES. 

★vrtiat  is  the  principal  religion  in  Uruguay? 


*could  you  please  indicate  the  principal  rivers  in 
Argentina? 

I  DON'T  UNDERSTAND  YOUR  WORD  "DDICATE.”  I’LL  GIVE  YOU 
ANOTHER  CHANCE. 

★the  principal  rivers  in  Argentina  are  ? 

THE  PRINCIPAL  RIVERS  IN  ARGENTINA  ARE  THE  RB5  EE  LA  PLATA, 
THE  PARANA,  .  .  . 


Figure  18.  Example  Dialogue  Between  a 

Student  (*)  and  SCHOLAR  (CAPS) 
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Note  first  that  the  student  is  able  to  query  the 
systepi  by  typing  a  question  in  place  of  a  response.  SCHOLAR 
had  asked  what  Brazil  was.  The  student  simply  responded  by 
asking  SCHOLAR  to  tell  him  something  about  Peru.  He  was  not 
specific  about  what  it  was  he  wanted  to  know  about  Peru.  The 
system's  natural  language  capabilities  allowed  it  to  recover 
from  an  unexpected  response,  comprehend  the  message,  and  be 
able  to  formulate  an  intelligible  answer.  To  get  the  infor¬ 
mation,  it  had  to  access  the  semantic  network.  Note  that 
the  system's  output  is  similar  to  the  internal  representa¬ 
tion  stored  in  the  network.  In  the  event  that  the  student's 
query  was  not  understood,  the  system  asked  the  student  for 
a  rephrasing  of  the  question.  The  system  obviously  only 
has  a  limited  vocabulary  and  can  only  communicate  in  a 
subset  of  the  English  language.  The  student  would  soon 
be  able  to  restrict  his  vocabulary  however,  as  he  gained 
experience  with  the  system. 


c 


l. 


The  purpose  behind  a  system  like  SCHOLAR  is  to  develop 
an  executor  program  which  could  transcend  various  subject- 
matter  domains.  The  executor  for  the  teaching  of  South 
American  geography  for  example  could  be  used  to  teach 
European  geography.  In  a  sense,  the  executor  is  a  model 
of  the  instructor,  while  the  knowledge  domain  represented 
in  the  semantic  network  is  a  model  of  the  expert.  The  set 
of  declarative  facts  represents  the  knowledge  of  the  expert 
which  the  student  is  to  acquire.  At  any  point  in  training 
the  student  is  modeled  as  a  subset  of  those  facts.  The 
executor,  or  model  instructor  is  to  present  new  informa¬ 
tion  or  facts  from  the  expert's  set  which  are  not  included 
in  the  student's  set.  Further,  the  model  instructor  is  to 
analyze  and  diagnose  errors  and  misconceptions  in  order  to 
ascertain  the  status  of  the  concepts  in  the  network.  The 
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4  04 

following  is  an  example  taken  from  Carbonell  (1970»  P.199). 

"Suppose  a  student  has  been  told  that  the 
language  in  Argentina  Is  Spanish,  but  when 
asked  about  the  language  in  Buenos  Aires, 
responds:  •Portuguese.’  Three  hypotheses 
can  now  be  made  about  the  student's  miscon¬ 
ception.  He  may  have  forgotten  what  the 
language  in  Argentina  is,  he  may  not  know 
that  Buenos  Aires  is  in  Argentina,  or  he 
may  not  be  able  to  draw  the  inference  that 
the  language  in  a  country  is  usually  the 
language  in  the  cities  located  in  that 
country." 


With  follow-up  questions  to  the  student,  the  executor  can 
reduce  its  uncertainty  about  the  causes  of  the  student's 
error  and  present  remedial  information  or  represent  the  con¬ 
cept,  which  it  had  presumed  was  learned,  as  being  in  an 
unlearned  state  and  return  to  it  later. 


A  more  recent  effort  funded  by  ONR  was  the  develop¬ 
ment  of  the  PLOW  tutor  and  is  reported  by  Gentner  and 
Norman  (  1 977)  ]  ^Norman  and  Gentner  (1978)]"*®  Norman 
( 1979) » and  Gentner  (1979)1^^  The  FLOW  tutor  represents 

its  knowledge  domain  as  schemata  and  prototypes  (Bobrow, 

141 

Kaplan,  Kay,  Norman,  Thompson  and  Winograd,  1977* 

Minsky,  1975  J)  as  described  earlier.  The  structural 
units  themselves  are  commands  and  statements  of  the  FLOW 
programming  language  instead  of  geographical  facts  as  in 
SCHOLAR.  The  FLOW  tutor  works  much  like  SCHOLAR  in  that 
the  knowledge  base  is  an  "expert"  and  the  student  can  be 
characterized  as  an  incomplete  collection  of  independent 
schemata  all  wanting  to  "fulfill"  themselves.  Associated 
with  each  schema  is  some  internal  intelligence  with  com¬ 
pletion  as  its  goal.  Coordinating  the  independent 
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structures  is  an  agenda.  The  agenda  is  a  list  of  incomplete 
schemata  petitioning  and  waiting  for  attention  in  order  to 
complete  their  parts. 

1 42 

Brown,  Burton  and  Zdybel  (1973)  developed  a  system 
called  SOPHIE  which  tutored  students  in  electronic  trouble¬ 
shooting.  The  student's  trouble-shooting  hypotheses  were 
compared  with  that  of  a  model  of  the  expert.  The  knowledge 
was  represented  in  procedural  form  rather  than  a  network. 

The  logic  of  the  system  was  to  move  the  student  in  the 

direction  of  the  model  of  the  expert.  More  recently.  Brown 

1 43 

and  Burton  (1977)  J  developed  a  system  called  BOGEY,  based 
on  the  building  of  a  procedural  model  for  the  development 
of  a  student's  arithmetic  skills. 

BIP-II,  developed  at  Stanford,  is  another  network- 
based  system  designed  to  teach  skills  in  BASIC  programming. 
The  knowledge  representation  is  described  as  a  Curriculum 
Information  Network  (CIN).  An  earlier  version  of  a  CIN- 
based  system  (BIP-l)  is  described  by  Barr,  Beard  and 
Atkinson  (1976)^^  with  the  BIP-II  enhancements  described 
by  Vescourt,  Beard,  Gould,  and  Barr  ( 1977)1^^  In  BIP-II 
the  CIN  was  developed  around  skills  rather  than  a  network 
of  declarative  facts,  and  utilized  additional  types  of 
linkages  between  nodes.  The  curriculum  of  BASIC  state¬ 
ments  was  first  organized  into  a  network  called  BASICNET . 

145 

An  example  portion,  taken  from  Wescourt  et.  al.  is 
shown  in  Figure  19.  The  nodes  themselves  represent 
conceptual  entities  regarding  BASIC  statements  while  the 
links  between  nodes  represent  relationships  between  the 
entities.  The  nodes  and  links  were  then  coded  in  a  list 
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K  CONTROL  STRUCTURE^ 

UNCONDITIONAL - - 

/  /  \  -  * 

C  - _ -rrlFTHEN 


*  CONDITIONAL 


"IF"  BOOLEAN 

NEXPR  REL 

/ 1  \  /!\ 


— \c 

"THEN"  linenum 


> 

NLIT  ■ 


’NEXPR 

— - - ►  NVAR 


FORNEXT 

^  X 

FOR  « - ■»  NEXT 


Figure  19.  A  Portion  of  BASICNET  Where  the  Links 

Between  Modes  Represent  J£ind,  Component, 
j^rdneas ,  etc. 


notation  for  the  computer.  Examples  of  such  listings 
would  bes 

(CONTROLSTRUCTURE  K  (UNCONDITIONAL  CONDITIONAL)) 

(CONDITIONAL  K  (IFTHEN  FORNEXT)) 

( IFTHEN  C  (IF  BOOLEAN  "THEN”  LIMENUM)  H  (FORNEXT)) 

From  these  three  listings  from  three  different  levels  in 
the  network,  it  can  be  seen  how  the  conceptual  entities 
can  be  represented. 

In  addition  to  the  BASICNET,  a  second  network  of 
skills  was  constructed  which  reflected  the  first  network. 

As  an  example,  a  w conditional  branch"  skill  (number  42  in 
the  listing)  was  represented  as  follows; ■ 

(SK042  (IFTHEN  (BOOLEAN  .  (NEXPR  .  NLIT) (NEXPR  .  NVAR)))) 
This  was  a  listing  of  a  skill  in  using  IF  THEN  statements 
between  a  numeric  literal  and  numeric  variable  with  the 
relation  unspecified.  Other  instances  of  essentially  the 
same  skill  could  then  be  described  relative  to  the  first. 

As  an  example,  skill  46  was  the  same  as  42  except  that  the 
relation  was  between  two  numeric  variables  instead  of  a 
numeric  variable  and  a  numeric  literal.  It  was  represented 
as; 


(SK046  (SK042  (NEXPR  .  NVAR))) 
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Note  that  the  listing  of  47  refers  to  42.  Skill  42  along 
with  its  other  MIFTHENM  instances  comprised  in  a  skill  set 
of  which  there  were  ten  such  skill  sets  in  all. 

Within  a  skill  set,  relations  between  pairs  of  skills 
were  established  by  referring  back  to  the  BASXCNET.  In 
the  following  example: 

(SK042  H  (SK044  SK046)  A  (SK047)  P  SK003)  ) 

Skill  46  (and  44)  is  judged  to  be  harder  than  42,  asdiown 
in  the  link  between  NLIT  and  NVAR  at  the  bottom  of  Figure 
19 •  Note  further  that  skill  42  is  said  to  be  analogous  to 
47  with  skill  3  as  a  prerequisite.  These  latter  two  rela¬ 
tionships  or  links  are  not  shown  in  Figure  19  because  of 

its  brevity.  The  list  of  linkages  between  skills  in  LISP 

1 45 

notation  comprised  what  Wescourt  et.al.  ^termed  the 
SKILLSNET. 

The  learning  of  the  skills  was  represented  as  a 
five-state  learning  process,  the  states  being  (l)  unseen, 

(2)  seen  but  experienced  trouble,  (3)  marginal,  (4)  unseen 
but  rated  as  easy  to  learn,  and  (5)  learned.  Briefly  the 
task  selection  logic  was  to  create  a  set  of  skills  occupying 
the  lowest  learning  states,  remove  the  skills  having 
unlearned  prerequisites,  and  then  select  tasks  (programming 
problems)  that  involve  the  designated  skills.  Since  each 
of  the  selected  tasks  may  vary  as  to  their  ratio  of  learned 
to  unlearned  skills  of  which  they  are  comprised,  the  actual 
task  selected  would  depend  on  the  history  of  the  progression 
of  the  particular  student. 

BIP-II  is  of  interest  not  only  in  terms  of  the  way 
knowledge  or  skills  were  represented  by  also  by  the  types 
of  linkages  between  nodes  and  the  procedure  for  generating 
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programming  tasks  for  the  student.  The  links  not  only 
represented  semantic  links  but  also  analogies,  generali¬ 
zations,  prerequisites,  and  relative  difficulties.  Thus 
tasks  generated  to  build  new  knowledge  could  take  into 
account  the  nature  of  the  knowledge  already  acquired. 

For  example,  the  printing  of  string  literals  can  be  pre¬ 
sented  as  being  analogous  to  the  printing  of  numeric 
literals  with  which  the  student  may  already  be  familiar. 
The  importance  of  presenting  new  information  in  terms 
of  old,  has  been  discussed  in  previous  sections  and  by 
Norman  and  his  colleagues. 

As  mentioned  previously,  the  training  systems  pre¬ 
sented  thus  far  base  their  student  model  on  a  representa¬ 
tion  of  the  expert.  They  view  the  student  as  a  set  of 
facts  or  skills  which  is  a  subset  of  those  presumably 
possessed  by  the  expert.  Goldstein  and  parr  (1977)^^ 
have  termed  such  models  as  "overlays"  to  emphasize  that 
the  characterization  of  the  student  is  a  derivative  of 
the  expert.  In  their  view,  these  overlays  suffer  from 
the  lack  of  any  representation  of  a  maturing  process. 

That  is  the  conceptual  entities  or  skills  which  the  stu¬ 
dent  is  to  acquire  is  represented  in  the  nature  and  re¬ 
fined  form  of  the  expert.  As  an  example,  when  the  stu¬ 
dent  is  first  introduced  to  the  concept  of  linear  regres¬ 
sion,  he  has  a  rather  simplistic  view  of  it.  Later  he 
learns  that  the  concept  can  be  generalized  to  non-linear 
regression.  As  Norman  (1978),J  puts  it,  the  student 
goes  through  periods  of  restructuring  and  tuning.  The 
importance  of  this  is  that  the  concept  of  linear  regress 
aion  that  the  student  acquires  is  only  an  immature  version 
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of  the  concept  held  by  the  expert.  Thus  the  network  repre¬ 
sentation  of  the  knowledge  domain  ought  to  represent 
epistomelogical  considerations. 

1 47 

Goldstein  (1979)  1  presents  a  tutor  for  the  game  of 

Wumpus  called  WUSOR-II.  It  evolved  from  the  WUSOR-I 
version  reported  by  Stansfield,  Carr,  and  Goldstein  (1976)1 
The  Wumpus  game,  for  those  not  familiar,  is  a  game  wherein 
the  player  is  placed  somewhere  in  a  warren  of  caves.  His 
goal  is  to  go  blindly  from  cave  to  cave  seeking  to  slay  the 
Wumpus  while  avoiding  the  dangers  of  pits,  bats,  and  of 
course,  the  Wumpus  itself.  The  player  can  make  inferences 
about  the  contents  of  a  cave  by  hearing  the  squeaks  of  the 
bats,  smelling  the  Wumpus,  and  feeling  a  draft  from  the 
pits.  Adding  to  this  his  memory  for  past  caves  he  has 
tried,  his  knowledge  then  is  a  set  of  rules,  procedures, 
and  logical  inferences  that  he  must  learn. 

The  distinction  of  WUSOR— II  is  that  it  employs  what 
Goldstein  calls  a  "genetic  graph"  to  represent  knowledge. 

The  idea  of  the  genetic  graph  is  that  it  attempts  to  cap¬ 
ture  the  evolutionary  process  by  which  a  mature  set  of 
procedures  evolve.  In  Figure  20  it  can  be  seen  that  the 
nodes  representing  the  rules  are  linked  by  relationships 
of  evolution.  In  the  example,  all  the  rules  shown  evolve 
from  rule  A  which  was  learned  initially.  Rule  AB  is  said 
to  be  a  refinement  of  rule  A  while  rule  ABC  is  a  version 
that  is  refined  still  further.  Rules  ADD.  AdD.  and  adD 
can  be  viewed  as  more  general  (or  more  specific)  versions 
of  rule  AD,  and  are  analogous  to  each  other. 

One  advantage  of  the  genetic  graph  is  that  it  repre¬ 
sents  the  student's  knowledge  at  various  stages.  With 
this  representation  the  system  would  not  present  rule 
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Figure  20.  A  portion  of  a  Genetic  Graph  Showing  Geneti 
Links  of  Refinement  (r),  Generalization/ 
Specification  (G/s)  and  Analogy  (a) 


ABC  without  first  going  through  the  versions  (A  an  i  AB)  at 
the  first  two  levels.  The  difference  between  this  and  the 
prerequisite  links  shown  in  BIP-II  is  that  rules  A,  AB,  and 
ABC  are  all  essentially  the  same  rule,  differing  only  in 

refinement,  whereas  prerequisites  refer  to  a  mandatory  order 
relationship  between  different  rules.  An  example  of  the 

evolution  of  a  rule  might  be  the  case  where  a  teacher  tells 
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the  student  the  primitive  form  of  the  spelling  rule  "i. 
before  and  later  modifies  it  with  exceptions  such  as 
MjL  before  e  except  afier  £.w  Thus  the  genetic  graph  gives 
the  system  information  as  the  maturity  level  with  which 
a  rule  should  be  given. 

A  second  advantage  is  that  it  would  provide  the  sys¬ 
tem  with  a  means  by  which  it  can  present  new  information 
in  terms  of  previously  learned  rules.  Take  for  example 
rule  ADD  in  Figure  20.  This  rule  could  be  presented  to 
the  student  as  a  generalization  of  rule  AD,  as  being  analo¬ 
gous  to  rule  adD  or  being  analogous  to  rule  AdD  providing 
that  these  other  rules  were  already  known.  Thus  the 
genetic  graph  provides  the  potential  for  a  very  powerful 
means  by  which  the  system  can  make  use  of  prior  knowledge 
in  generating  multiple  styles  by  which  a  rule  could  be 
presented. 

As  can  be  seen,  the  knowledge  representation  system 
(KRS)  is  basic  to  a  generative  system  with  mixed  initia¬ 
tive  capabilities.  Only  if  a  system  can  generate  tasks, 
questions  or  responses  to  student  queries  from  an  appro¬ 
priately  represented  knowledge  domain,  can  it  possess  the 
flexibility  required  to  provide  truly  individualized 
instruction  as  would  a  human  tutor.  The  way  in  which  the 
knowledge  is  represented  is  important  for  several  reasons: 

(1)  Since  the  ultimate  goal  of  the  system  is  to  move 
the  student  in  the  direction  of  the  representation  of  the 
expert,  the  KRS  should  be  sufficiently  rich  to  have  incor¬ 
porated  all  the  essential  features  of  the  expert's  knowlege 
and  skills. 

(2)  An  understanding  of  the  representation  of  knowledge 
in  the  human  (both  student  and  expert)  is  essential  for  the 
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development  of  pedagogical  techniques  for  the  building, 
restructuring  and  tuning  of  knowledge  structures. 

(3)  As  portrayed  in  Goldstein's  genetic  graph,  the 
KRS  should  be  sufficient  to  represent  evolutionary  stages 
especially  if  the  methodology  is  to  be  extended  to  the 
training  of  time-sharing  tasks. 

(4)  Multiple  modes  concept  presentation  based  on  the 
student's  current  knowledge  should  be  implied  from  the  KRS, 
as  in  the  example  of  presenting  a  concept  as  an  analogy, 

or  a  generalization  of  another  concept. 

(5)  Finally,  the  KRS  should  be  complete  enough  to 
allow  the  system  to  generate  output  in  natural  language 
format . 

Space  limitations  do  not  allow  for  a  detailed  discus¬ 
sion  of  the  more  technical  aspects  of  representing  know¬ 
ledge.  Readers  currently  engaged  in  the  development  of 
training  systems  should  consult  the  sources  cited  along 
with  Bott  (1979)66  and  Bobrow  and  Winograd  ( 1 977 ) ^  ^ 
concerning  a  knowledge  representation  language. 

NATURAL  LANGUAGE  CAPABILITIES.  The  natural  language  capa¬ 
bilities  of  an  intelligent  system  are  inexplicably  corre¬ 
lated  with  the  representation  of  knowledge.  Most  of  all 
the  well-known  systems  cited  thus  far  have  limited  know¬ 
ledge  domains  such  as:  electronic  trouble-shooting  in 
SOPHIE,  a  programming  language  in  FLOW  and  BIP-II,  arith¬ 
metic  skills  in  BOGEY,  and  tactical  and  inference  skills 
in  WUSOR.  The  result  of  limited  domain  is  limited  actions, 
concepts  and  constrained  terminology  and  phraseology.  This 
would  certainly  be  true  for  aircraft-controller  type 
training  systems. 
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Burton  and  Brown  (1979)^^  describe  four  properties 
that  are  imperative  for  a  natural  language  processor: 
efficiency,  habitability,  the  ability  to  shape  the  student's 
phraseology  and  tolerance  for  ambiguity.  Efficiency  refers 
to  the  system's  ability  to  parse  the  student's  query,  under¬ 
stand  it,  deduce  or  calculate  an  answer  for  him  followed  by 
the  generation  of  the  response.  All  this  must  be  done  under 
a  two  second  response  time  to  avoid  affecting  performance 
(see  Miller,  1968  3  )  and  well  under  that  to  operate  within 
a  real  time  aircraft  controller  environment.  Habitability 
refers  to  the  system's  ability  to  understand  and  operate  in 
a  large  enough  subset  of  the  natural  language  to  be  useful. 
System  capacity  will  prohibit  the  size  of  the  language  sub¬ 
set,  both  now  and  in  the  near  future,  but  the  subset  should 
be  large  enough  to  handle  a  large  portion  of  the  student's 
self- initiated  responses  within  the  constraints  of  the 
subject  matter. 

Consistent  with  the  requirement  of  habitability,  is 
the  requirement  that  the  system  be  able  to  reduce  the  stu¬ 
dent's  variations  in  phraseology  throughout  the  training 
sessions  to  be  able  to  reduce  the  amount  of  times  the  sys¬ 
tem  has  to  ask  the  student  to  rephrase  the  question.  The 
fourth  requirement  is  that  the  system  be  aware  of  ambiguity 
or  multiple  deep  structures  for  a  given  surface  structure 
as  in  the  phrase,  "Was  the  Wumpus  believed  to  have  been 
killed  by  Fred?"  Here  the  student  could  be  asking  if  Fred 
killed  the  Wumpus  or  if  Fred  believed  that  the  Wumpus  had 
been  killed.  Recognizing  the  ambiguity,  the  system  could 
take  steps  to  resolve  it.  Not  recognizing  the  ambiguity, 
the  system  could  possibly  generate  an  answer  in  reference 
to  one  of  the  two  meanings  and  get  into  a  series  of  mis¬ 
understandings  with  the  student. 
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To  achieve  these  goals  Brown  and  Burton  (1979)^^ 
discuss  the  use  of  semantic  grammars  as  opposed  to  the  older 
syntactic  grammars  (see  Simmons,  1970  for  a  review  of 
early  language  processors) .  Using  semantic  processing 
they  report  that  the  system  requires  less  than  200  milli¬ 
seconds  cpu  per  question  from  the  student.  Part  of  the 
success  in  processing  rate  comes  from  the  use  of  semantic 
information  during  parsing  for  predictive  purposes.  For 
example  after  a  noun  phrase  and  veito  jfarase  have  been  parsed 
in  a  top-down,  lef t-to-right  manner,  the  object  of  a  subse¬ 
quent  prepositional  phrase  can  be  anticipated.  This  antici¬ 
pation  can  reduce  the  amount  of  any  lexical  search  any 
grammatical  ambiguity,  and  aid  in  the  determination  of  ref¬ 
erents  for  pronouns .  This  predictive  process  is  of  interest 
in  its  own  right  as  a  result  of  the  possibility  of  the 
human  aircraft-controller  using  a  similar  preprocessing 
and  predictive  process  in  the  anticipation  of  events  on 
the  display. 

Many  of  the  intelligent  systems  discussed  Implement 

their  language  processing  capabilities  with  LISP  (see 

Teltelman,  197^  J  )•  LISP  is  compilable  and  was  used 

initially  with  SOPHIE.  Later  a  compilable  version  of  a 

semantic  augmented  transition  network  (ATN)  was  developed 

153 

at  Bolt  Beranek  and  Newman  (see  Burton,  1976  JJ)  and 
used.  The  AIN  process  (a  notion  first  introduced  by 
Chomsky)  has  advantages  over  LISP.  Space  does  not  per¬ 
mit  a  full  discussion  of  the  technical  details  between 
the  two  here,  but  the  sources  are  quite  worthy  of  further 
scrutiny.  Future  developments  of  the  use  of  ATNs  at 
Bolt  Beranek  and  Newman  and  other  sources  of  research  in 
natural  language  processing  should  be  monitored  closely. 
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SECTION  IV 

INSTRUCTOR  MODEL:  PRELIMINARY  CONSIDERATIONS 


A  Pull  development  of*  an  instructor  model  is  premature 
at  this  time.  As  we  will  attempt  to  shoy,  there  are  certain 
technological  gaps  which  need  to  be  filled  befoie  a  complete 
model  of  the  instructor  for  automated-speech  based  training 
systems  can  be  constructed.  In  the  sections  to  follow,  we 
would  like  to  make  use  of  the  basic  research  developments 
just  reviewed  in  order  to  propose  directions  which  the 
instructor  model  development  should  take.  In  outlining 
these  characteristics,  the  cbaxacteristics  of  the  traditional 
human  instructor  need  to  be  kept  in  mind  for  reference 
purposes.  Finally  the  technological  gaps  precluding  the 
current  implementation  of  such  a  model  need  to  be  discussed. 

CHARACTERISTICS  OF  THE  HUMAN  INSTRUCTOR 

The  instructors  of  aircraft  controllers  are  skilled 
controllers  themselves,  and  may  in  some  cases  be  pilots. 

Thus  the  first  characteristic  we  would  have  to  make  would 
be  that  the  instructor  in  most  any  subject  matter  carries 
an  internal  knowledge  base.  Secondly,  he  possesses  know¬ 
ledge  about  relationships  between  the  event-driven  tasks. 

He  knows  that  having  the  students  work  on  skills  pertaining 
to  certain  concurrent  tasks  causes  confusion  early  in  train¬ 
ing.  Thus  in  addition  to  a  factual  knowledge  base,  he  may 
have  a  knowledge  base  regarding  the  dynamic  effects  of  task 
concurrence. 

Thirdly,  the  instructor  has  experience  in  instructional 
presentation  techniques.  As  an  example,  it  is  often  heard 
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that  certain  experienced  instructors  have  excellent  tech¬ 
niques  or  skills  in  getting  information  across.  Some 
instructors  have  developed  effective  skills  in  the  use  of 
analogies  for  making  the  student  attend  to  specific  features, 
while  other  instructors  may  have  developed  techniques 
requiring  students  to  verbalize  differences  between  similar 
events  as  a  means  of  getting  them  to  attend  to  the  relevant 
features  that  would  support  differentiation.  Thus  through 
experience,  the  instructors  have  acquired  a  repertoire  of 
tutorial  skills  and  procedures. 

The  fourth  characteristic  is  that  the  instructor's 
natural  language  capabilities  support  mixed-initiative 
dialogue  during  the  course  of  instruction.  He  is  able  to 
generate  statements  for  the  student  as  well  as  to  under- 

t 

stand  and  to  respond  to  student-generated  queries. 

As  a  fifth  characteristic,  it  should  be  pointed  out 
that  the  human  instructor  has  a  limited  capability  and  an 
intuitive  desire  for  diagnosis.  When  an  error  is  made  the 
instructor  will  be  found  asking  the  student,  "Why  did  you 
do  that?"  or,  "What  was  your  thinking  al;  the  time?"  The 
instructor  may  even  manipulate  the  student's  task  somewhat 
in  an  effort  to  diagnose  the  student's  problem.  The 
diagnosis  is  then  followed  by  remediation.  If  it  were 
ascertained  that  the  student  erred  because  of  a  misconception, 
the  instructor  then  attempts  to  redefine  the  concept. 

A  sixth  characteristic  is  that  the  instructor  has  an 
intuitive,  if  not  formal,  set  of  performance  measures  by 
which  he  is  constantly  assessing  the  student's  progress. 

These  measures  may  be  informal  and  unsystematic,  but  he 
still  makes  use  of  them,  even  if  they  are  only  comprised 
of  facial  expressions  of  confusion  or  insight. 
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The  seventh  characteristic  is  that  he  imputes  a 
subjective  utility  distribution  over  the  points  in  the 
knowledge  and  skills  domain.  Certain  parts  of  the  domain 
are  thought  to  be  more  important  than  others  and  this 
affects  the  distribution  of  his  instructional  efforts. 

The  order  and  extent  of  these  instructional  efforts 
determine  his  eighth  characteristic,  the  design  of  the 
curriculum.  Though  the  instructor  possesses  the  intelli¬ 
gence  to  simply  react  to  student  initiatives,  he  generally 
takes  the  initiative  during  most  of  the  instruction  and 
imposes  a  curricular  structure  on  himself.  This  curriculum 
is  a  product  of  past  experience  and  serves  as  an  aid  in 
ensuring  the  breadth  of  his  instructional  coverage  as  well 
as  a  memory  aid  on  preconceived  order  constraints  on  topics. 

The  curriculum  then  serves  as  an  external  supplement  to 
his  internal  representation  of  the  knowledge  domain. 

A  final  characteristic  is  that  the  instructor  is  con¬ 
tinually  revising  his  approach.  Through  additional  exper¬ 
ience  he  revises  his  curriculum,  his  subjective  utility 
estimates,  performance  measures,  diagnostic  skills  and 
remediation  techniques  and  his  whole  pedogogical  approach 
in  general. 

THE  INSTRUCTOR  MODEL 

Consider  for  the  moment  that  there  are  no  limits  in 
research  and  development  resources,  and  that  development 
monies  exist  in  great  quantities.  With  no  constraints,  we 
could  then  ask  ourselves  what  we  would  like  to  see  in  an 
automated  model  of  the  instructor.  What  are  the  possible 
features  that  an  intelligent  training  system  could  incor¬ 
porate,  based  on  the  reviews  of  the  current  developments 
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in  basic  research  just  presented?  Following  an  outline  of 
the  possibilities,  we  can  then  come  back  to  reality  and 
examine  the  effects  of  cost  and  resource  considerations 
in  development . 

At  the  outset,  we  could  say  that  an  intelligent  system 
with  considerable  flexibility,  would  be  desirable.  The 
system  could  follow  a  curriculum  overall  but  w>uld  be  locally 
generative  within  segments  of  the  total  syllabus.  It  would 
possess  a  student  model  of  developing  knowledge  and  skill 
structures,  and  diagnostic  capabilities  with  generative 
techniques  for  remediation.  Natural  language  communication 
would  be  needed  to  support  mixed- iniative  discourse  with 
the  student  as  well  as  an  articulate  interface  with  human 
supervisors.  Finally,  the  system,  with  or  without  human 
intervention,  should  be  able  to  modify  itself  as  it  gains 
experience  from  repeated  samples  of  student  population. 

For  illustration  purposes,  let  us  assume  that  the 
model  instructor  to  be  developed  is  to  be  incorporated  in 
an  automated  training  system  designed  for  our  composite 
controller  task.  It  wiJ 1  be  recalled  that  the  composite 
task  was  a  hypothetical  aggregation  of  the  three  roles  of 
PAR  controllers,  LSOs  and  AICs  contrived  for  explanatory 
purposes . 

KNOWLEDGE  REPRESENTATION  SYSTEM.  For  maximal  flexibility 
in  a  generative  system,  the  knowledge  domain  needs  to  be 
represented.  The  work  cited  earlier  shows  much  promise 
for  portions  of  our  domain.  Factual  information  which  may 
be  in  the  curriculum,  such  as  shown  in  Figure  21  could  be 
represented  easily  enough  in  a  semantic  network.  The 
representaion  of  factual  information  would  be  only  a  part 
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Figure  21 . 


Illustration  of  the  PAR  Schemata  as  They 
Might  be  Represented  in  a  Semantic  Network 


of  the  need  however.  Highly  formalized  procedural  infor¬ 
mation  (such  as  a  "check-in"  procedure),  and  the  procedures 
requiring  some  initiative  on  the  part  of  the  student,  (such 
as  required  of  the  AIC's  decision  as  to  what  information 
the  pilot  needs  before  he  intercepts  the  bogey)  could  best 
be  represented  in  a  genetic  graph.  This  would  allow  the 
evolutionary  stages  of  the  student's  grasp  of  the  procedure 
or  role  to  be  represented.  Trend  calls  regarding  the  azi¬ 
muth  could  be  represented  as  analogies  of  the  trend  calls 
on  the  elevation.  Position  calls  are  static  forms  of  trend 
calls.  The  more  complex  procedural  concepts  of  the  AIC 
could  be  represented  in  their  various  levels  of  maturity. 
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Hie  KRS  developments  cited  earlier  indicate  that  the 
representation  of  factual  and  procedural  information  is 
not  only  feasible  but  desireable.  The  research  on  compe¬ 
tition  for  central  resources  however,  points  out  an  impor¬ 
tant  need.  The  adaptive  logic,  as  will  be  discussed  below, 
of  an  automated-speech  based  training  system  will  need  to 
be  involved  in  resource  budgeting  during  the  training  of 
complex  concurrent  tasks .  Previously  we  had  spoken  of 
three  relationships  between  pairs  of  tasks:  symbiotic, 
competitive,  and  independent.  Let  us  assume  a  simple  net¬ 
work  of  tasks  which  represent  those  relationships  as  shown 
in  Figure  22.  Let  us  further  assume  for  illustration  that 
we  could  quantify  task  demand  for  central  resources  as  a 
percentage  figure,  both  for  tasks  executed  in  isolation  and 
c  oncurr ent ly . 


Figure  22.  Network  of  Five  Tasks  with  the  Links 

Representing  Symbiotic  (S),  Competitive  (C) 
or  Independent  (I)  Pairwise  Relationships 
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As  can  be  seen  in  Figure  22,  tasks  b  and  c  could  be 
introduced  as  two  independent  tasks.  In  isolation,  let  us 
assume  that  for  a  given  level  of  performance  on  each  task 
b  and  £  require  50$  and  70$  of  the  resources  respectively. 
If  truly  independent,  then  foreseeably  the  two  tasks  could 
be  executed  concurrently  with  no  more  than  70$  of  the  cen¬ 
tral  resources  required.  Increasing  the  investment  of 
resources  would  increase  the  performance  levels  of  both 
tasks.  Tasks  A  and  B  are  said  to  have  a  symbiotic  rela¬ 
tionship.  If  each  required  50$  in  isolation,  a  symbiotic 
relationship  implies  that  they  might  require  only  30% 
concurrently.  As  an  example  trend  calls  might  augment 
position  calls  and  vice  versa.  Thus  saying  "below  glide- 
path"  followed  by  the  trend  statement  "coming  up"  sets  up 
an  expectation  for  the  next  position  call  to  be  "slightly 
below  glidepath."  Thus  trend  calls  and  position  calls 
could  augment  each  other  to  the  extent  that  the  central 
resources  demanded  for  their  joint  requirement  could  even 
be  less  than  their  separate  demands  singly. 

Tasks  D  and  C  are  said  to  have  a  competitive  relation¬ 
ship.  If  both  demand  70$  of  the  available  resources  sepa¬ 
rately,  then  their  competitive  relationship  would  imply 
that  their  joint  performance  might  demand  more  than  the 
simple  sum  of  their  separate  demands— 1 60$.  Since  the 
student  can  give  only  100$  in  total,  one  or  the  other  or 
both  of  the  two  tasks  must  suffer  with  the  student  in  this 
overload  situation.  In  this  situation,  if  a  student  were 
to  vary  his  allocation  of  resources  invested  in  D  and  C 
between  the  extremes  of  a  70$/ 30$  to  30$/70$  split,  a  POC 
of  the  type  shown  in  the  left  panel  of  Figure  5 ,  would  be 
produced. 


130 


NAVTRAEQUIPCEN  79-C-0085-1 


The  adaptive  logic  of  the  instructor  model  could  do 
one  of  three  things  at  this  point.  First,  the  student  could 
be  trained  in  isolation  on  task  C  until  it  reaches  automa- 
ticity  or  becomes  more  resource  efficient  as  is  shown  in 
Figure  8.  Then  when  10$  of  the  resources  in  isolation  pro¬ 
duce  the  same  required  level  of  performance  as  before, 
task  D  could  be  introduced.  Again  because  of  their  competi¬ 
tive  relation,  their  cost  for  concurrence  may  be  more  than 
the  sum  of  their  separate  costs,  say  90$  >  70$  +  10$.  But 
because  of  the  advance  training  on  C,  the  joint  demand  of 
the  two  tasks  is  now  manageable. 

A  second  possibility  for  the  adaptive  logic  of  the 
instructor  model  would  be  to  reduce  the  joint  demand  of 
the  two  tasks.  This  could  be  done  by  adjusting  the  adap¬ 
tive  variables  on  one  or  both  of  the  tasks  (e.g.  the  pace 
could  be  slowed  by  adjusting  the  airspeed  of  the  approach¬ 
ing  aircraft).  This  would  be  a  more  likely  instructional 
strategy  to  take  if  the  training  of  tasks  in  isolation 
presents  a  real  compromise  in  simulation  fidelity. 

It  may  be  that  both  previous  approaches  represent 
undesireable  compromises,  and  that  the  overload  situation 
simply  cannot  be  avoided.  The  joint  performance  will  simply 
have  to  suffer  until  practice  eventually  makes  the  tasks 
more  resource  efficient.  In  that  case,  the  adaptive  logic 
takes  on  the  duties  of  resource  budgeting.  By  telling  the 
student  to  attend  to,  or  concentrate  on,  one  task  more  than 
the  other,  the  system  is  manipulating  the  student’s  subjec¬ 
tive  performance  utilities,  and  thus  the  indifference  curves. 
Assuming  resource  allocation  acquisition  functions  of  the 
sort  shown  in  Figure  12,  telling  the  student  to  concentrate 
more  on  a  particular  task  would  increase  the  rate  in  which 
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that  task  would  be  mastered  (become  resource  efficient).  As 
discussed  previously,  the  job  of  the  instructor  model  would 
be  to  solve  for  the  optimal  allocation  of  resources  (the 
turnpike  solution")  during  the  joint  training  of  C  and  D. 

In  order  for  the  instructor  model  to  have  the  capabilities 
just  described,  the  pairwise  relationships  regarding  resource 
requirements  between  tasks  would  need  to  be  represented. 
Further  the  representation  of  those  relationships  would  need 
to  be  modified  throughout  the  course  of  training.  The  peri¬ 
odic  estimation  of  pairwise  relationships  would  be  no  small 
feat  and  will  be  discussed  with  performance  measurement 
issues . 

ADAPTIVE  LOGIC.  As  first  described  by  Kelly  ( 1 969 ) 1 
adaptive  training  has  been  used  in  various  trainers,  des¬ 
cribed  in  numerous  reports,  and  thus  will  not  be  reviewed 
here  (see  Atkinson,  1976  J  ).  Briefly  the  idea  is  that  per¬ 
formance  during  training  can  be  held  constant  by  varying  an 
adaptive  variable  (presumably  increasing  the  difficulty  of 
the  task  as  learning  progresses.)  In  the  present  situation 
with  multiple  concurrent  tasks,  the  adaptive  logic  problem 
is  a  bit  more  complicated.  It  will  be  recalled  that  Navon 
and  Gopher  (1979)  7  make  a  distinction  between  task  demand 
and  task  difficulty.  It  will  be  in  this  sense  that  we  will 
be  using  task  demand  as  a  potential  adaptive  variable. 

The  usage  of  the  semantic  network  in  generative  CAI 
format  as  described  above,  has  as  its  goal  the  building  of 
knowledge  structure.  Using  the  terminology  of  Norman 
(  1 978 ) T information  is  provided  and  manipulated  in  the 
accretion  and  restructuring  phases  of  learning.  Adaptive 
training,  on  the  contrary,  is  used  not  for  the  building  of 
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structures  but  rather  for  refinement  or  tuning  of  process. 

Once  the  student  "knows"  vhat  it  is  he  is  supposed  to  do, 
he  must  practice  the  procedure  until  it  becomes  resource 
efficient  or  automatic.  The  building  of  knowledge  struc¬ 
tures  or  schemata  we  will  refer  to  as  the  tutorial  function 
of  the  system,  while  the  tuning  and  extensive  practice  will 
be  referred  to  as  the  adaptive  function  of  the  system. 

The  goal  of  the  adaptive  function  is  the  unitizing  of 
a  structure  so  that  it  becomes  automatic,  or  resource  effi¬ 
cient  as  shown  in  Figure  8.  At  first  the  demands  of  the 
task  (or  an  aggregation  of  tasks  as  discussed  in  the  pre¬ 
vious  section)  are  reduced  so  that  the  student  is  not  over¬ 
loaded.  As  the  resource  efficiency  increases,  the  task 
demands  are  increased.  When  the  task  demands  reach  their 
prescribed  level,  a  new  task  may  be  added  to  the  aggregate, 
and  the  demands  reduced  again. 

The  demands  of  the  tasks  can  be  controlled  in  several 
ways.  The  presentation  rate  of  events  is  an  obvious  first 
choice  for  an  adaptive  variable.  The  speed  of  the  approach¬ 
ing  aircraft  can  be  presented  unrealistically  slow  at  first 
so  that  each  schema,  whether  it  be  conceptual  or  motoric, 
can  be  processed  to  completion.  If  task  demands  are  too 
high,  the  student  will  experience  many  trials  in  which  a 
sequence  is  not  completed.  As  discussed  in  previous  sections, 
it  is  important  for  the  unitizing  process  that  processing 
sequences  be  highly  practiced  in  their  entirety.  The  speed 
of  the  aircraft  could  be  increased  eventually  beyond  even 
the  fastest  normal  approach  rate  in  order  to  force  the  stu¬ 
dent  into  automatic  processing.  Just  how  far  to  go  with  this 
should  be  under  the  control  of  the  intelligence  of  the  system. 


133 


NAVTRAEQUIPCEN  79-C-0085-1 


A  second  choice  for  so  adaptive  variable  would  be  the 
synchrony  of  events .  Since  the  beginning  student  is  charac¬ 
terized  by  resource  consuming  controlled  processing,  which 
is  serial  in  nature,  it  would  be  obvious  that  successive 
presentation  of  events  rather  than  simultaneous  presenta¬ 
tion  of  events  would  ease  his  burden.  In  the  PAR  task,  the 
aircraft  would  deviate  only  from  the  glidepath  while  staying 
well  within  tolerance  on  the  azimuth  and  vice  versa.  Devia¬ 
tions  would  commence  only  after  the  completion  of  hand-off 
procedures.  On  the  AIC  task,’  the  introduction  of  a  bogey  is 
timed  to  occur  only  after  the  check— in  procedure  has  had 
time  to  be  completed.  Jinks  occur  when  nothing  else  is 
demanding  the  subject's  resources.  Later  in  the  training 
of  the  current  aggregation  of  skills,  scenarios  are  generated 
by  the  system  in  which  everything  seems  to  happen  at  once, 
or  at  least  in  close  temporal  proximity.  Thus  the  synchrony 
of  events  in  the  generated  scenarios  would  be  controlled  by 
the  adaptive  logic.  When  the  aggregate  tasks  are  mastered, 
new  requirements  are  introduced  and  the  adaptive  logic  begins 
generating  scenarios  with  successive  tasks  again.  In  order 
for  the  system  to  possess  the  intelligence  to  generate 
scenarios  with  appropriate  synchrony,  some  index  of  the 
current  resource  demands  of  the  event  are  required.  The 
estimation  problem  will  be  discussed  later. 

A  third  adaptive  variable  for  consideration  would  be 
task  scheduling.  The  system  may  alternate  between  tutorial 
and  adaptive  training  modes.  The  tutorial  mode  may  be 
entered  as  a  new  task  is  added  to  the  composite,  followed  by 
an  adaptive  training  period  in  which  the  new  task  is  prac¬ 
ticed  in  conjunction  with  the  others  already  established. 

The  adaptive  feature  comes  in  the  choice  of  task  (sequencing) 
and  the  timing  of  the  introduction  of  the  task.  The  choice 
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of  task  would  be  based  on  the  representation  of  concur¬ 
rence  properties  (symbiotic,  competitive  or  independent) 
with  the  previously  established  tasks  as  discussed  in  the 
last  section.  The  choice  of  when  to  introduce  the  next 
task  should  be  controlled  by  an  optimizing  routine  such  as 
introduced  by  Chant  and  Atkinson  (1973)»  If  the  new 
task  is  introduced  too  early,  the  last  task  may  not  be 
resource  efficient  enough  yet,  and  the  student  would  end 
up  in  an  overload  situation,  precluding  rapid  acquisition 
of  the  new  skill.  If  the  task  is  introduced  too  late,  the 
new  skill  may  be  acquired  rapidly  enough  but  time  was 
wasted  by  it  not  being  introduced  earlier  when  the  previous 
task  became  resource  efficient. 

NATURAL  LANGUAGE  CAPABILITIES.  In  observing  students  on  the 
PAR  trainer,  it  was  observed  that  they  had  a  tendency  to 
want  to  turn  to  their  human  instructor  and  ask  for  help  or 
advice,  or  even  to  comment  that  things  were  getting  con¬ 
fusing.  This  desire  is  quite  natural,  and  is  evident  in 
the  frustration  produced  by  nonarticula te  systems  wherein 
the  student  cannot  take  any  initiative.  On  some  systems, 
such  as  the  present  nonautomat ed  PAR  trainer,  the  real  time 
scenario  may  be  frozen  long  enough  for  tutorial  intervention 
by  the  human  instructor.  This  would  be  a  desireable  feature 
on  an  automated  system  as  well.  The  approach  of  an  aircraft 
could  be  frozen  long  enough  for  the  student  to  query  the 
system  as  to  what  he  should  do  in  the  current  situation. 

The  system  could  also  intervene  when  its  diagnostics  indi¬ 
cated  that  errors  in  performance  were  caused  by  structural 
problems  rather  than  resource  limitations.  Upon  entering 
the  tutorial  mode,  its  natural  language  capabilities  could 
query  the  student  in  further  diagnosing  his  misconceptions. 
Following  the  tutorial  intervention,  the  adaptive  training 
function  could  be  resumed. 
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A  second  function  of  the  natural  language  capabilities 
would  be  to  provide  an  articulate  interface  with  a  human 
supervisor  in  charge  of  training.  The  supervisor  could 
query  the  system  regarding  the  current  status  of  a  certain 
student.  He  could  ask  for  the  misconceptions  of  the  student 
or  the  current  status  of  the  resource  efficiency  of  certain 
task  requirements.  The  supervisor  or  author  of  the  system, 
during  possible  debugging  sessions,  could  query  the  system 
as  to  why  a  particular  task  was  selected  at  some  specific 
point  in  training.  The  system  could  respond  in  a  natural 
language  form  suitable  for  a  training  supervisor  not  familiar 
with  the  usual  form  of  computer  output.  Wescourt  et  al. 
(1977)^^  discuss  articulate  interfaces  along  with  interactive 
debugging  aids  and  their  potential. 

DIAGNOSTICS.  The  complete  system  would  be  organized  around 
a  syllabus.  It  would  intermittently  alternate  between  its 
tutorial  and  adaptive  functions  either  as  prescribed  by  the 
syllabus,  or  as  demanded  by  the  dynamics  of  the  student 
system  interaction.  Following  introductory  material,  the 
syllabus  might  begin  with  tutorials  in  perceptual  learning. 
Pairs  of  "snapshot”  type  images  could  be  displayed  on  the 
CRT  such  as  would  be  desireable  in  the  training  of  percep¬ 
tual  discriminations  required  of  the  LSO.  The  perceptual 
training  could  be  followed  by  tutorials  required  for  the 
building  of  knowledge  structures  and  procedural  schemata. 
Interspersed  in  these  tutorials,  would  be  the  adaptive 
sessions  designed  to  fine  tune  each  of  the  procedural  schemata 
to  some  extent  before  the  next  tutorial.  As  the  student  ad¬ 
vances,  his  progress  is  monitored  by  the  updating  of  a  stu¬ 
dent  model.  The  updates  on  the  student  model  require  a 
constant  diagnostic  activity  on  the  part  of  the  system. 
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The  appropriate  form  of  remediation  is  inexplicably 
tied  to  the  diagnosis  of  the  cause  of  performance  failures. 
The  intelligence  of  the  system  must  first  be  able  to  distin¬ 
guish  between  errors  of  structure  and  process.  The  error 
may  have  been  caused  by  resource  limitations  ("Too  many 
things  happened  at  once!"  is  the  comment  from  the  students) 
or  the  error  may  have  been  caused  by  a  misconception  (the 
student  wouldn't  have  gotten  the  correct  response  even  if 
he  had  unlimited  time  to  respond) .  As  discussed  in  an 
earlier  section,  errors  of  the  first  type  imply  adjustments 
in  the  adaptive  variable(s)  whereas  errors  of  the  second 
type  imply  the  need  for  tutorial  intervention  to  restructure 
schemata.  In  order  to  distinguish  between  structure  and 
process  errors,  the  task  demands  or  scenarios  may  be  adjusted 
for  diagnostic  purposes.  If  a  student  had  given  an  improper 
heading,  the  system  could  generate  the  next  scenario  with 
characteristics  similar  to  the  one  in  which  the  heading  error 
had  occurred,  but  with  a  slower  pace  or  with  the  critical 
events  presented  successively  rather  than  simultaneously. 

In  other  words,  adjustments  in  the  adaptive  variables, 
designed  to  reduce  the  demands  of  the  task,  would  be  imple¬ 
mented.  If  the  error  persists,  then  tutorial  intervention 
would  be  required,  otherwise  the  error  would  be  judged  as 
process-limitation  precipitated  requiring  additional  adjust¬ 
ments  of  the  adaptive  functions. 

The  diagnostics  of  the  system  should  also  be  capable  of 
discriminating  between  causes  of  errors  relative  to  process¬ 
ing  stages.  Consider  as  an  example  the  situation  wherein  an 
LSO  gives  the  incorrect  advisory,  "A  little  more  power"  when 
in  fact  the  pitch  angle  is  too  high  and  the  aircraft  is 
already  slightly  above  glides lope.  The  correct  advisory  may 
have  been  to  bring  the  nose  down  which  would  result  in  a 
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slight  increase  in  airspeed  and  a  settling  down  on  the 
glidepath.  The  incorrect  advisory  may  have  been  due  to 
the  student's  inability  to  perceive  and  distinguish  between 
the  fine  gradations  of  pitch  angle  as  shown  in  Figure  13 
(implying  deficient  perceptual  schemata)  or  if  the  student 
was  process  limited  at  the  time,  he  may  not  have  had  time 
to  visually  scan  the  silhouette  for  its  subtle  features 
(implying  that  the  perceptual  schema  has  not  yet  been  suf¬ 
ficiently  unitized).  On  the  other  hand,  the  pitch  may  have 
been  perceived  correctly  but  the  student  has  not  yet  estab¬ 
lished  the  correct  cause  and  effect  relationship  between 
"more  power"  and  the  specific  pitch  angle  present  (implying 
deficiencies  in  conceptual  schemata) .  Other  causes  could 
also  be  present  such  as  incorrect  expectancies  due  to  his 
lack  of  experience.  The  point  is  that  the  different  causes 
(perceptual,  attentional,  conceptual,  etc.)  will  require 
different  forms  of  remediation:  additional  perceptual  dif¬ 
ferentiation  training,  tutorials  on  the  joint  effect  of 
power  and  pitch,  etc. 

This  quality  of  remediation  requires  an  intelligent 
diagnostic  componant  capable  of  manipulating  task  demand 
parameters  to  this  end.  An  additional  means  of  diagnosis 
would  be  self-diagnosis  by  the  student  whereupon  the  system 
uses  its  natural  language  capabilities  to  interrogate  the 
student.  For  a  system  without  natural  language  capabilities 
a  menu  of  common  complaints  (e.g.  "didn't  expect  it  to,..," 
"too  many  things  happening,"  "didn't  know  what  to  do,"  etc.) 
could  be  displayed.  With  an  appropriate  choice  of  items, 
a  student  could  adequately  introspect  sufficiently  for 
realistic  diagnostic  purposes. 
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STUDENT  MODEL.  All  the  examples  of  CAX  systems  described 
previously  have  some  method  for  representing  the  student 
in  his  progress  through  the  course.  It  will  be  recalled 
that  one  form  of  this  representation  is  to  view  the  student 
as  a  growing  subset  of  skills  or  knowledge  units.  The 
system  keeps  a  record  of  vhich  skills  have  been  mastered 
and  which  are  left  to  be  acquired.  In  its  simplest  form 
then,  each  skill  or  knowledge  structure  can  occupy  one  of 
only  two  states:  learned  or  unlearned.  A  more  extensive 
representation  is  illustrated  by  the  BIP  II  system.  It 
will  be  recalled  that  in  this  system  the  individual  skills 
were  viewed  as  occupying  one  of  five  states  ranging  from 
"unseen"  to  "learned."  Thus  acquisition  was  seen  as  a 
multi-state  process  wherein  instructional  strategy  was 
dependent  upon  state  occupancy. 

1 1|7 

Goldstein  (1979)  '  in  his  genetic  graph,  provided  a 

different  technique  for  representing  the  progress  of  a 
skill.  Instead  of  representing  the  skill  at  a  single  loca¬ 
tion  in  the  network  wherein  the  skill  occupied  one  of  two 
or  one  of  five  learning  states,  he  represented  the  skill 
at  several  locations  in  the  graph  wherein  the  various  loca¬ 
tions  represented  evolutionary  stages.  His  reasoning  was 
that  it  is  not  really  the  same  skill  at  those  various  loca¬ 
tions  but  rather  qualitatively  different  and  increasingly 
refined  versions  of  the  skill. 

What  we  would  require  in  our  ideal  system  is  possibly 
both  forms  of  representation:  first,  a  single  location 
multi-learning  state  representation  for  structural  units 
that  do  not  undergo  change  as  they  are  learned  (see 

4  e  e 

Atkinson,  1976  D:>)  and  secondly,  a  multi-location  repre¬ 
sentation,  such  as  in  the  genetic  graph,  when  the  skill  or 
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structural  unit  undergoes  some  transformation  or  refinement 
as  learning  progresses.  Ve  would  also  desire  thirdly,  a 
continuous  representation  of  the  strength  of  a  skill  or 
schema.  This  last  requirement  would  yield  information  re¬ 
garding  the  degree  of  automaticity  of  a  skill  which  should 
be  represented.  Finally,  we  would  also  desire,  to  the  ex¬ 
tent  possible,  that  the  representation  of  the  progress  of  a 
structural  unit  be  formalized  in  stochastic  form,  which 
would  allow  the  adaptive  logic  in  the  system  to  optimize 
its  instructional  decisions  (see  Chatfield  and  Gidcumb, 
197737). 

PERFORMANCE  MEASUREMENT.  In  the  preceding  discussions,  it 
is  apparent  that  performance  measurement  is  needed  for  both 
the  adaptive  and  tutorial  functions.  The  adaptive  logic 
requires  performance  measurement  for  the  real  time  adjust¬ 
ment  of  task  demand,  task  scheduling  and  event  synchrony 
in  the  generation  of  scenarios.  The  tutorial  function 
obviously  needs  learning  state  information  regarding  the 
structures  in  the  network  for  the  selection  of  tutorial 
strategies  and  dialogues.  The  system  could  not  present 
concept  B  as  an  analogy  or  extension  of  concept  A  if  con¬ 
cept  A  is  in  a  primative  or  unlearned  state. 

Task  scheduling  requires  that  the  system  possess  infor¬ 
mation  regarding  the  resource  efficiency  of  a  task  or  schema 
and  its  resource  sharing  relations  with  other  tasks  as  shown 
in  Figure  22.  Ideally  we  would  like  to  record  the  changes 
in  the  POC  for  each  pair  of  tasks,  and  a  resource  allocation 
function  for  each  task.  Further  we  would  like  to  record 
the  changes  in  the  POC  and  resource  allocation  functions 
(see  Figure  8)  as  training  progresses.  We  can  conclude 
from  the  outset  however,  that  attaining  and  revising 
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that  much  information  would  be  an  impossible  task  for  any 
performance  measurement  technique ,  even  if  the  measurements 
were  possible.  Thus  the  POCs  between  pairs  of  tasks  may 
have  to  be  approximated  by  discrete  states  (competition, 
independence,  and  symbiosis). 

Crude  measures  or  inferences  of  selected  points  in  the 
resource  allocation  functions  for  tasks  might  be  possible, 
however.  To  do  this,  some  variations  on  the  secondary  task 
and/or  probe  RT  methodology  could  be  used.  As  an  example, 
consider  the  case  where  the  student  has  just  mastered  the 
time-sharing  of  tasks  A,  B,  and  C.  Ve  know  that  the  "A,  B, 

Cw  composite  is  now  resource  efficient  because  an  artificial 
signal  cancellation  task  (artificial  in  terms  of  it  having  no 
value  in  subsequent  job-related  duties)  was  added  to  the 
display  simply  for  measurement  purposes.  Whenever  the  sig¬ 
nal  is  presented,  the  student  must  immediately  cancel  it  by 
pushing  a  button.  The  reaction  time  required  for  such  a 
response  could  be  a  sensitive  measure  of  processing  load, 
even  more  so  than  accuracy  measures  (see  Hunt,  Lansman  and 

IQ. 

Wright,  1979  ).  When  the  RTs  are  sufficiently  short  for 

the  maximum  demand  on  the  "A,  B,  composite,  the  fourth 
task  (d)  would  be  added  and  the  task  demand  for  the  new 
"A,  B,  C,  D"  composite  reduced.  If  the  RTs  lengthen  beyond 
tolerance  limits, the  demands  would  be  decreased  further. 

Thus  the  signal-cancellation  measure,  contrived  simply  for 
measurement  purposes,  could  act  as  the  performance  measure¬ 
ment  required  in  the  adaptive  logic. 

If  an  artificial  task  such  as  the  one  just  described  is 
unacceptable,  then  adaptations  to  the  secondary  task  method¬ 
ology  could  be  devised.  As  task  D  is  added  to  the  A,  B,  C 
composite,  some  decrement  to  the  performance  of  A,  B,  or  C 


141 


NAVTRAEQUIPCEN  79-C-OO85-I 


or  some  combination  thereof  should  be  realized.  The  exact 
source  of  the  decrement  would  depend  on  the  student's  sub¬ 
jective  indifference  functions.  Examples  of  these  measures 
would  be  such  things  as  the  degradation  of  speech  quality, 
response  latencies  to  events  such  as  the  jinking  of  a  bogey, 
and  the  accuracy  of  computed  headings.  Two  problems  arise 
with  these  "natural"  measures.  The  first  is  the  discontinuity 
or  irregularity  in  their  occurrence  and  occasions  for  measure¬ 
ment.  The  second  is  that  certain  of  the  accuracy  and  speech 
degradation  measures  may  be  hard  to  quantify  in  a  way  which 
would  be  sensitive  enough  for  adaptive  control.  Assuming 
that  with  some  amount  of  research  and  development  effort 
appropriate  measures  can  be  defined,  resource  demands  for  a 
given  performance  level  and  resource  efficiency  could  be 
measured  at  some  points  in  the  resource  allocation  function, 
albeit  the  measurement  may  be  indirect. 

Some  of  the  more  conceptual  forms  of  schemata  or  even 
perceptual  schemata  may  not  be  directly  associated  with  any 
continuous  performance  measurement.  A  PAR  student  may  have 
a  misconception  of  the  details  and  implications  of  an  air¬ 
craft  making  sin  approach  with  hydrolic  system  failure.  This 
misconception  cannot  be  continuously  measured  and  can  only 
be  measured  upon  an  error.  Even  then  the  diagnostics  of  the 
system  must  be  able  to  identify  the  error  as  being  due  to 
the  misconception  rather  than  excessive  processing  demands. 
This  is  where  the  natural  language  capabilities  eind  the 
intelligence  of  the  system  can  be  beneficial.  The  non¬ 
observability  of  learning  states  has  been  a  problem  for 
psychologists  for  some  time  (see  Greeno,  1 968  ]  "^Greeno  and 
Steiner,  1968; ^Levine  and  Burke,  1972^^).  The  problem 
lies  in  the  reliance  on  inferences  since  by  definition  the 
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transitions  between  states  cannot  be  observed.  An  intelli¬ 
gent  system  may  not  only  manipulate  events  for  diagnostic 
purposes  but  also  enter  the  tutorial  mode  and  query  the 
student.  This  would  be  a  tremendous  help  when  parameter 
estimation  and  unidentifiable  states  are  a  problem  in  the 
student  model. 

SELF  MODIFICATION  PROPERTIES.  As  the  instructor  model 
obtains  additional  data  from  the  students,  it  should  be 
able  to  benefit  from  that  experience  and  become  a  more 
efficient  training  system.  Examples  of  such  benefits  are 
discussed  by  Atkinson  and  Paulson  ( 1 972) ]  “^Atkinson 
(  1976)  ]  ^^and  Chant  and  Atkinsbn(  1978)  The  improvements  in 
these  examples  cited,  come  mainly  through  increased  pre¬ 
cision  in  parameter  estimation  for  the  student  models. 

One  benefit  comes  from  the  increase  in  precision  in  the 
optimization  algorithms,  which  are  dependent  upon  the 
parameter  estimates.  A  second  benefit  comes  from  qualita¬ 
tive  changes  in  the  student  model  based  on  the  additional 
data.  As  an  example,  Atkinson  and  Paulson  ( 1 972) ^  "^report 
using  the  Random  Trials  Increments  (RTl)  model  (Norman, 
(l964)^^as  the  model  of  the  student.  As  is  characteristic 
of  the  RTI  model,  variation  in  the  parameter  can  change 
its  qualitative  nature  from  that  of  an  all-or-none  process 
to  an  incremental  process  as  extremes.  By  letting  the 
parameters  be  estimated  empirically,  the  most  representative 
mix  of  these  two  models  can  be  found.  Further  the  represen¬ 
tation  improves  with  added  data  from  additional  students. 

This  principle  of  self-adjustment  through  empirically 
derived  estimates  controlling  a  process,  could  be  generalized 
to  portions  of  the  system  other  than  just  the  student  model. 
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SECTION  V 


SYSTEM  DEVELOPMENT 

The  characteristics  of  the  instructor  model  just  outlined, 
were  developed  without  regard  to  any  practical  limitations 
in  terms  of  time  and  resources.  These  characteristics  repre¬ 
sent  the  ideal  prototype  in  terms  of  technologies  available 
or  anticipated.  Consider  for  a  moment  the  position  of  the 
applied  contractor  wh<?  must  develop  a  complete  training  sys¬ 
tem  of  the  type  just  described.  The  sections  to  follow  rep¬ 
resent  ventures  into  the  world  of  reality  with  which  the  con¬ 
tractor  must  deal:  a  dommentary  on  the  practicality  of 
implementing  the  ideas  we  have  gleaned  from  the  products  of 
basic  research. 

CONTRACTING  CONSIDERATIONS 

For  research  and  development  (R&D)  in  the  areas  previously 
described,  the  Navy  uses  five  levels  of  contracting  designa¬ 
tions.  These  represent  levels  of  effort  ranging  from  basic 
research  to  operational  implementation.  Simply  put,  these 
levels  represent  R&D  stages  through  which  a  voice-interactive 
training  system  would  pass  in  going  from  the  idea  stage  to 
implementation  and  production.  The  various  levels  also 
represent  sources  of  fluids,  and  the  regulation  of  the  kinds 
of  R&D  activities  that  are  to  take  place  at  each  stage.  The 
R&D  levels  defined  by  the  Defense  Acquisition  Regulations 
(DAR),  formerly  the  Armed  Service  Procurement  Regulations 
(ASPR),  are  as  follows:  6.1  level— research j  6.2  level- 
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exploratory  development;  6.3  level— advanced  development; 

6.4  level— engineering  development;  and  finally  the  last 
stage  of  operational  implementation  where  the  system  goes 
into  production  and  is  placed  into  the  fleet. 

Funds  for  the  6.1  level  of  effort  would  be  invested  in 
basic  research  projects  which  would  produce  a  technology 
(such  as  the  research  developments  in  the  representation  of 
knowledge)  which  might  be  applicable  to  a  broad  range  of 
applied  development  efforts.  The  6.2  designation  is  de¬ 
signed  to  provide  funds  for  exploratory  and  feasibility 
studies.  As  an  example  of  such  an  effort,  Feuge  et  al . 
(1974)  produced  a  feasibility  study  for  the  PAR  trainer, 
GCA-CTS.  In  this  effort,  he  explored  the  functional  require 
ments  of  a  PAR  controller,  outlined  a  laboratory  training 
system,  and  did  the  initial  evaluation  of  its  potential. 
Similar  efforts  have  been  carried  out  by  private  contrac¬ 
tors  for  the  feasibility  and  design  of  an  auto-adaptive 

K 

LSO  training  system  (Hooks,  et  al.  1978"/  and  a  demonstra¬ 
tion  training  system  for  the  AIC . 

The  contract  for  the  development  (6.4  level)  of  the 
GCA-CTS  was  awarded  to  Logicon,  Inc.,  a  private  contractor. 
In  this  effort,  the  initial  design  by  Breaux  and  others  was 
further  developed  and  tested.  Once  the  system  was  fully 
developed,  enhanced  and  tested,  it  was  taken  to  a  PAR 
training  location  in  Memphis,  Tennesee  for  refinement  in 
an  operational  environment  before  going  into  production. 

An  example  of  this  development  process  is  shown  in  Figure  23 
As  can  be  seen,  development  efforts  from  the  6.2  level 
downward  are  organized  around  a  product,  i.e.,  a  training 
system.  Only  at  the  6.1  level  dQes  the  thematic  units 
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Figure  23.  Present  Organizational  Structure  of  R&D  Effort 

cross  the  specific  training  system  efforts.  Thus,  as  an 
example,  when  the  GCA-CTS  was  originated,  it  made  use  of 
the  developing  technology  in  computerized  speech  recogni¬ 
tion.  The  research  efforts  in  speech  recognition  however, 
transcend  a  single  training  system  and  can  be  applied  to 
a  broad  spectrum  of  systems. 

ONR  is  in  the  business  of  supporting  basic  research  at 
the  6.1  level,  although  it  does  have  some  6.2  monies.  The 
Naval  Training  Equipment  Center  is  primarily  engaged  in  the 
development  of  training  equipment  at  the  6.2  through  the 
6.4  .Levels,  although  it  does  have  some  access  to  6.1  monies. 

The  idea  behind  this  organization  is  that  the  basic  research, 
as  described  in  the  previous  sections,  is  to  be  used  by  the 
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6.2  level  contractors  in  developing  their  products.  However, 
as  is  depicted  in  Figure  23,  we  propose  that  there  is  at  some 
times  a  gap  between  the  6.1  and  6.2  and  the  6.4  and  production 
levels.  There  is  usually  continuity  in  personnel  and  contrac¬ 
tors  working  on  the  development  of  a  training  system  from  the 
6.2  to  6.4  levels?  but  because  of  the  difference  between  the 
nature  of  the  6.1  and  6.2  levels  of  effort,  there  is  no  conti¬ 
nuity  between  the  respective  contractors.  The  result  is 
that  the  6.2  level  contractors  are  often  not  aware  of  6.1 
achievements,  and  the  6.1  level  contractors  are  not  aware 
of  the  6.2  needs. 

TECHNOLOGY  GAPS 

When  a  bid  is  solicited  from  contractors,  the  govern¬ 
ment  is  interested  in  firms  that  not  only  have  sufficient 
physical  plant  facilities  and  capital  to  carry  out  the 
contract,  but  who  also  possess  the  personnel  with  the 
expertise  to  build  a  system  which  incorporates  the  latest 
of  technologies.  The  contract  however  usually  does  not 
allow  for  much  time  or  resources  on  the  part  of  the  con¬ 
tractor  to  "figure  out"  (6.1  effort)  how  to  build  the 
system  (i.e.  do  literature  searches  or  carry  out  6.1 
or  6.2  level  R&D) .  Thus,  with  the  constraints  of  time 
and  resources,  the  contractor  is  limited  to  incorporating 
only  those  technologies  relatively  well  developei  which 
require  minimal  resources  to  apply. 

The  products  of  basic  research  described  in  the  previous 
section  are  in  various  developmental  stages  relative  to 

implementation  in  a  system  such  as  our  ideal  prototype.  The 

< 

most  remote  stage  in  terms  of  implementation  would  be  the 
case  wherein  the  technology  has  just  been  identified  as 
being  a  need.  The  need  may  have  only  recently  arisen  and 
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been  identified  as  a  result  of  a  contemporary  development* 

One  step  closer  to  implementation  would  be  the  situation 
wherein  the  technological  gap  or  need  as  been  partially 
filled  by  theory .  Here  basic  theory  has  been  formulated 
which  describes  a  potential  solution  for  the  identified 
need,  but  an  actual  working  demonstration  (such  as  imple¬ 
mentation  in  a  CAI  package)  has  not  been  developed.  A 
step  closer  yet,  would  be  the  case  that  the  solution  has 
been  refined  and  developed  sufficiently  to  have  been  incor¬ 
porated  into  a  working  demonstration  system.  The  last  stage, 
representing  a  minimum  gap,  would  be  the  case  wherein  the 
solution  or  technology  is  in  the  implementation  stage.  Here 
the  technology  has  been  incorporated  into  a  system  of  suffi¬ 
cient  similarity  to  the  system  being  developed  that  only 
minor  modification  and  development  would  be  required.  This 
situation  would  be  likely  to  arise  when  a  firm  has  previously 
developed  similar  systems.  These  four  stages  of  need,  theory,  { 

demonstration,  and  implementation  are,  of  course,  not  discrete. 
Additionally,  various  components  of  technologies  may  con¬ 
currently  be  in  different  stages.  Still,  these  stages  should 
aid  us  in  reviewing  the  status  of  the  technologies  identified 
as  desirable  for  an  automated  instructor  model. 

KNOWLEDGE  REPRESENTATION.  Semantic  networks  and  the  genetic 
graph  are  both  in  the  demonstration  stage.  Though  the  tech¬ 
nology  exists,  considerable  effort  would  be  required  for 
implementation.  An  idea  of  the  extent  of  the  required  effort 
can  be  obtained  by  looking  at  the  demonstration  efforts. 

As  an  example,  Wescourt  et  al .  (1977)^+^  describes  the  devel¬ 
opment  of  both  BIP-I  and  I31P-II.  BIP-I  was  essentially 
developed  from  the  theory  stage  and  could  represent  a  crude 
upper  limit  on  development  effort.  BIP-II  could  be  consid¬ 
ered  a  lower  limit  in  that  second  effort  would  have  presum¬ 
ably  benefitted  from  the  experience  on  the  first.  The  two  ( 
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versions  would  have  shared  the  same  knowledge  domain  as 
well  as  much  of  the  same  technical  staff.  Had  a  knowledge 
representation  system  been  developed  for  the  GCA-CTS 
(however  most  of  the  demonstration  projects  cited  were  still 
under  development  at  the  inception  of  GCA— CTS)  the  trainer 
would  have  taken  longer  to  develop  and  been  more  costly. 
However,  later  systems  such  as  those  under  development  for 
the  LSOs  and  AICs  would  have  benefitted  from  the  prior 
experience . 

The  development  of  a  system  which  would  represent  task 
concurrence  properties  could  be  described  as  being  back  at 
the  "need"  stage  or  the  beginning.  The  implications  of  this 
type  of  representation  emanate  from  the  studies  cited  and 
this  report.  The  development  of  this  technology  would 
require  a  6.1  level  of  effort,  as  it  is  doubtful  that  it 
could  be  incorporated  into  a  developing  system.  We  would 
propose  that  the  development  of  this  kind  of  a  representa¬ 
tion  system,  wherein  the  effects  of  task  concurrence  and 
developing  resource  efficiency  is  represented,  begin  with 
simulation  runs.  This  approach  would  accelerate  the  devel¬ 
opment  of  the  adaptive  logic  algorithms  which  would  use 
this  information  as  well  as  provide  an  evaluation  of  dif¬ 
ferent  forms  of  representation.  Wescourt  et  al.  (1977)*^ 
discuss  their  usage  of  simulation  to  test  the  sufficiency 
of  BIP,  but  we  would  advocate  using  it  earlier  in  the  step- 
by-step  development  of  tne  logic. 


NATURAL  LANGUAGE  INTERFACE.  The  use  of  an  articulate  inter¬ 
face  for  tutorial  intervention,  allowing  the  student  to 
initiate  tutorial  dialogue,  and  the  use  of  natural  language 
capabilities  to  assist  a  human  instructor  or  supervisor, 
would  be  an  extremely  powerful  development  for  the  future. 
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This  development  is  now  at  the  demonstration  stage ,  but 
there  are  some  constraints  in  terms  of  its  near  future 
implementation  in  the  instructor  model.  Sources  at  the 
Xerox  foundation  and  Bolt  Beranek  and  Newman,  Inc.,  where 
its  development  has  taken  place,  indicate  that  it  presently 
requires  a  large  system,  e.g,  PDP-10X,  which  would  be  too 
large  to  be  cost-effective  for  the  type  of  training  systems 
which  have  been  discussed.  Developments  in  this  area  should 
be  monitored,  however,  because  of  their  potential. 

ADAPTIVE  LOGIC  AND  PERFORMANCE  MEASUREMENT.  The  use  of 
adaptive  logic  and  performance  measurement  in  one  sense 
could  be  said  to  be  in  the  implementation  stage  in  that 
previous  systems  with  a  high  degree  of  similarity  have 
utilized  these  principles.  However,  some  of  the  ideas 
expressed  in  this  paper  would  require  some  development  and 
could  be  said  to  be  in  the  "need”  stage.  The  first  problem 
would  be  performance  measurement.  Measures  more  sensitive 
than  simple  error  frequency  would  be  needed  for  resource- 
efficiency  estimation.  This  in  fact  could  be  explored  to 
some  extent  by  the  firms  developing  the  system.  The  usage 
of  ancillary  measures  such  as  digit  cancellation,  and  dual 
task  latency  measures  in  conjunction  with  the  requisite 
adaptive  logic  in  event  rate,  task  synchrony,  and  task 
scheduling  should  be  developed.  This  would  require  a  series 
of  6.2  or  6.1  efforts  which  are  not  tied  by  a  specific 
training  system  end  product. 

A  portion  of  the  task  of  building  an  adaptive  logic 
and  measurement  system  based  on  a  resource  competition 
model  would  be  considered  as  basic  research,  and  thus 
funded  at  the  6.1  level.  The  remainder  of  these  concepts— 
development  specifically  for  controller  training  systems— 
would  be  too  specific  for  6.1  funding  possibly  and  should 
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be  carried  out  at  the  6.2  level,  perhaps  by  a  human  factors 
laboratory.  Firms  engaged  in  the  development  of  a  particular 
system  would  then  be  able  to  pick  up  the  technology  in  a 
ready-to-go  state. 

STUDENT  MODEL  AND  DIAGNOSTICS.  The  model  of  the  student  and 
the  requisite  diagnostics  need  to  be  developed  in  conjunction 
with  a  knowledge  representation  system.  The  current  repre¬ 
sentation  of  the  student  as  a  growing  subset  of  the  knowledge 
domain  is  a  technology  in  the  demonstration  stage.  The  addi¬ 
tional  representation  of  the  acquisition  or  building  of 
knowledge  structures  as  a  multi-state  process  is  also  at 
the  demonstration  stage.  Thus  in  the  current  development  of 
a  training  system,  the  designers  would  only  need  to  look  at 
the  development  of  previous  systems  such  as  BIP.  This 
development  would  probably  be  best  done  by  contractors  that 
would  later  be  building  similar  systems,  so  as  to  amortize 
the  cost  of  the  initial  development  in  modifying  the  techni¬ 
ques  of  one  of  the  prior  demonstration  projects. 

The  formalization  of  the  student  model  as  a  stochastic 
process  would  be  another  technology  at  the  demonstration 
stage  as  are  the  optimization  routines.  The  creation  of  a 
model  which  would  represent  resource  efficiency  however, 
would  be  best  described  as  being  in  the  theory  stage.  There 
are  numerous  strength  models  which  could  be  utilized  but 
considerable  effort  would  be  required  to  develop  such  a 
model.  The  effort  would  be  one  of  those  that  could  be 
either  6.1  or  6.2  efforts  depending  on  the  generality  of  the 
technology.  The  required  effort  would  probably  be  too 
extensive  to  be  included  in  a  system  development  contract* 

SELF-MODIFICATION  PROPERTIES.  The  technology  behind  self¬ 
modification  could  be  very  important  to  the  development  of 
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future  systems.  The  technology  is  only  beginning  and  would 
be  considered  to  be  in  the  theory  stage.  However,  this  is 
one  of  those  areas  that  could  be  supported  as  general  6.1 
research  as  a  part  of  the  intelligence  domain,  supported  as 
6.2  level  research  for  specific  techniques  that  could  be  con¬ 
ducted  by  human  factors  laboratories,  or  could  be  a  part  of 
a  systems  development  firms  own  R&D  program.  This  would  be 
one  case  where  it  might  be  cost-effective  for  a  firm  to  do 
its  own  R&D  in  that  this  technology  could  supplement  a  lack 
of  technology  in  other  areas.  As  an  example,  if  a  firm  were 
awarded  a  contract  for  a  training  system  in  an  area  where 
even  the  SMEs  may  have  little  insight  into  the  cognitive 
requirements  of  the  task,  the  development  of  a  flexible  and 
intelligent  system  with  self-modification  properties  could 
eventually  converge  on  a  set  of  rational  if  not  optimal 
algorithms  in  instructional  policy. 

MOTOR  THEORY.  A  major  focus  of  the  two  current  motor 
theories  enjoying  some  degree  of  attention  is  the  error 
detection  mechanism.  Combining  some  of  Adam's  recent  dis¬ 
cussions  on  new  directions  in  KR  research  it  would  be  bene¬ 
ficial  to  the  understanding  of  the  acquisition  of  the  error 
detection  mechanism  to  investigate  the  facilitation  of  error 
detection  (and  correction)  as  a  function  of  the  locus  of  KR 
during  training  trials.  To  the  extent  that  error  detection 
and  correction  may  be  considered  independent  processes,  one 
could  further  investigate  their  independent  development  as 
a  function  of  KR  location. 

The  HPM  as  presented  by  Marteniuk  and  as  reviewed  here, 
makes  a  major  distinction  between  open  and  closed  tasks.  In 
some  of  the  preceding  discussion,  it  was  pointed  out  that 
nomenclature  is  too  restrictive  to  be  applied  accurately  to 
all  of  the  component  tasks  of  the  trainer.  It  would  therefore 
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be  worthwhile  to  pursue  the  development  of  a  taxonomic  system 
more  versatile  than  the  dichotomous  ’’open  vs.  closed”  nomen¬ 
clature  which  is  currently  in  use.  Most  probably  there 
would  be  several  useful  dimensions  of  descriptors  that  would 
better  reflect  the  processing  and  output  characteristics  of 
the  task,  as  well  as  the  input  characteristics  as  is  currently 
done.  A  better  defined  task  (problem)  would  then  lead, 
hopefully,  to  a  more  accurate  solution. 

EXPECTATION  AND  AUTOMATIC ITY .  As  discussed  previously, 
there  was  some  question  as  to  the  degree  to  which  expecta¬ 
tions,  automaticity,  or  both  were  responsible  for  the  expert’s 
seeming  resource  efficiency.  This  would  be  important  as 
instructional  strategy  would  be  dependent  upon  the  distinc¬ 
tion.  Paradigm  techniques  for  the  investigation  of  this 
problem  were  suggested  earlier.  One  other  question  along 
similar  lines,  would  be  the  conditions  under  which  the 
training  of  automaticity  and  expectations  can  be  facilitated. 
Shiffrin  and  Schneider  speak  to  this  problem  when  they  refer 
to  consistent  mapping  and  varied  mapping  conditions,  category 
effects,  etc.  So  basic  research  has  produced  some  answers 
but  specific  recommendations  within  the  present  context  need 
to  be  made.  These  questions,  regarding  the  specificity  for 
the  development  of  the  current  training  systems,  are  best 
placed  in  the  6.2  research  domain.  They  concern  the  ques¬ 
tion  of  the  training  of  time-shared  tasks  in  general  and 
would  probably  be  more  appropriate  for  the  human  factors 
laboratory  rather  than  the  R&D  within  a  systems  development 
firm. 

PERCEPTION.  Obviously,  a  big  factor  in  the  development  of 
a  training  system  for  LSOs  is  going  to  be  the  development 
of  a  technique  for  identifying  the  visual  features  to  which 
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the  LSO  attends.  This  has  already  been  done  to  some  extent 
and  is  presently  incorporated  in  the  LSO  training  program. 
Additionally  the  technology  for  the  training  of  perceptual 
skills  needs  to  be  established.  These  technologies  would 
all  be  considered  to  be  at  the  theoretical  stage  of  develop¬ 
ment.  Most  of  these  efforts  could  be  funded  at  either  the 
6.1  or  6.2  level.  The  AFOSR  is  currently  funding  the 
development  of  feature  identification  techniques  as  mentioned 
previously.  The  technology  of  perceptual  training  such  as 
differentiation  training,  affordance  training,  etc.  can  be 
obtained  from  the  basic  literature  but  needs  to  be  directed 
to  the  issues  inherent  in  the  automated  training  of  con¬ 
trollers  (probably  a  6.2  level  of  effort).  The  specific 
identification  of  LSO  perceptual  features  would  have  to  be 
carried  out  as  one  of  the  initial  efforts  In  the  design  of 
the  training  system. 

INFORMATION  CROSSFEED 

PROBLEM.  As  in  most  technical  and  scientific  disciplines 
there  is  a  communication  problem  in  terras  of  the  dissemin¬ 
ation  of  large  amounts  of  technical  and  scientific  informa¬ 
tion.  The  effect  is  that  the  applied  scientists  developing 
the  training  systems  are  not  able  to  incorporate  recent 
advances  from  basic  research,  while  the  basic  researchers 
are  not  aware  of  the  applied  needs.  This  problem  is  acute 
in  that  a  good  portion  of  the  basic  research  reviewed  in  the 
present  report  is  funded  by  the  Navy  (ONR)  while  the  current 
needs  discussed  herein  also  emanate  from  the  Navy  (NAVTRA¬ 
EQUIPCEN).  As  a  part  of  the  contract  which  produced  this 
report,  the  senior  author  was  able  to  attend  meetings  re¬ 
garding  this  problem  as  well  as  to  discuss  possible  solutions 
with  personnel  from  both  the  applied  and  basic  domain.  From 
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these  discussions  it  was  concluded  that  the  overall  problem 
was  two-fold.  One  was  an  ’'awareness"  problem  and  the  other 
was  the  "state-of-the-technology 

AWARENESS.  It  is  of  course  difficult  to  describe  the 
characteristics  of  a  group  of  individuals  such  as  applied 
scientists  in  that  their  levels  of  awareness  in  the  areas 
reviewed  in  this  report  would  vary  in  terms  of  their  area 
of  expertise  and  the  area  of  their  current  projects.  We 
feel  it  safe  to  say  however,  that  most  have  only  marginal 
knowledge  of  the  current  ONR  programs  being  funded.  They 
may  know  the  general  thrust  of  ONR  funding  but  lack  the 
awareness  of  the  current  technological  products  coming  out. 

The  current  authors,  prior  to  this  review,  were  in  the  same 
state.  After  reviewing  the  programs  it  was  evident  that 
a  major  portion  of  the  basic  research  which  would  impact 
automated  training,  is  being  supported  by  ONR.  Thus,  if 
one  could  even  keep  abreast  of  these  programs,  part  of  the 
awareness  problem  could  be  solved.  For  the  benefit  of 
interested  readers.  Appendix  A  includes  a  synopsis  of  the 
ONR  programs  and  various  projects. 

The  information  summarizing  the  ONR  efforts  was  pro¬ 
vided  by  that  office  for  this  report,  but  the  information 
is  not  routinely  disseminated.  It  was  the  opinion  of 
several  scientists  that  it  would  be  beneficial  if  informa¬ 
tion  such  as  this  were  provided  to  the  applied  community. 

(The  form  of  the  information  may  need  to  be  revised  somewhat 
however.)  More  than  one  commented  when  shown  a  listing  and 
synopsis  of  the  ONR  programs,  that  it  was  not  readily  apparent 
how  some  of  the  products  of  basic  research  could  be  applied 
to  their  projects.  Thus  it  was  suggested  that,  not  only 
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would  regular  announcements  of  ONR  projects  be  helpful,  but 
the  abstracting  of  the  projects  could  be  organized  along 
the  lines  of  their  impact  on  applied  programs. 

As  can  be  seen  in  the  present  review,  the  literature 
that  would  impact  the  development  of  training  systems, 
whether  it  be  funded  by  ONR,  AFOSR,  National  Science 
Foundation  (NSF),  etc.,  is  varied  and  quite  extensive. 

No  one  resource  person  is  going  to  be  able  to  keep  abreast 
of  all  the  areas  of  perception,  cognition,  CAI,  motor 
learning,  speech  recognition,  artificial  intelligence, 
control  theory,  and  systems  programming.  Thus  a  periodic 
review  of  the  literature  would  be  required  of  various 
personnel . 

The  question  of  the  cost  of  keeping  abreast  of  the 
literature  must  be  addressed.  Though  it  is  not  a  major 
cost,  it  is  certainly  noticeable  in  a  competitive  bidding 
environment  where  the  government  wants  an  accounting  of 
all  activities.  The  systems  development  firms  themselves 
must  of  course  bear  the  cost  of  at  least  keeping  abreast 
of  the  basic  developments  that  were  listed  as  being  in  the 
demonstration  stage.  But  the  government  itself  should  bear 
some  of  the  cost  by  providing  dissemination  channels, 
conferences,  workshops,  etc.  This  it  already  does  to  some 
extent,  but  the  comments  received  were  that  an  overview 
and  translation  of  the  products  of  basic  research  and  their 
relevance  would  be  beneficial.  The  present  report  is  in 
part  an  attempt  by  the  government,  i.e.  the  Naval  Training 
Equipment  Center,  to  summarize  and  translate  some  of  the 
achievements  of  basic  research,  in  terms  of  its  relevance 
to  the  development  of  automated-speech-based  training  systems 
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STATE  OF  THE  TECHNOLOGY.  Simply  reviewing  and  summarizing 
the  literature  is  not  sufficient.  As  noted  in  a  previous 
section,  the  products  of  basic  research  are  in  various 
stages  of  completion  regarding  immediate  implementation. 

Some  are  only  in  the  demonstration  or  theory  stage  and  are 
likely  to  remain  there  until  an  applied  effort  is  initiated 
which  will  continue  the  development  in  the  direction  of  the 
training  systems.  Other  products  may  be  immediately  applied 
with  only  minimal  modification.  The  point  is,  however,  that 
the  technological  breakthroughs  require  some  developmental 
effort  to  be  applied. 

Awareness  is  only  a  part  of  the  problem.  ONR  could  dis¬ 
tribute  summaries  of  projects,  NAVTRAEQUIPCEN  could  sponsor 
periodic  reviews  of  the  literature,  and  workshops  on  current 
developments  could  be  held,  and  still  a  good  portion  of  the 
developments  would  not  be  implemented.  It  is  our  opinion 
that  the  reason  lies  in  the  nature  of  current  contracting 
practices . 

Informally  it  could  be  said  that  there  are  two  types 
of  contracts  that  are  let.  One  could  be  best  described  as 
a  technology  oriented  contract  where  the  effort  is  to  attempt 
to  solve  a  problem  by  executing  certain  agreed  upon  activi¬ 
ties.  The  outcome  of  the  effort,  like  research  in  any  dis¬ 
cipline,  is  uncertain.  The  contractor,  attacking  the  pro¬ 
blem  with  great  scientific  skills  and  executing  the  experi¬ 
mental  manipulation  in  the  grandest  of  style  cannot  guaran¬ 
tee  the  direction  of  the  data,  or  even  that  the  problem  will 
be  solved.  What  he  agrees  to  in  the  contract  is  to  expend 
an  agreed  upon  amount  of  resources  in  carrying  out  the 
general  approach  or  design  with  which  he  won  the  contract 
award . 
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The  second  type  of  contract,  may  be  referred  to  as 
product  oriented.  In  this  case,  the  contract  requires  an 
end  product,  or  a  solution  as  it  were,  for  the  efforts 
expended.  All  contracts  require  an  end  product  of  some 
kind,  even  if  it's  simply  a  report  on  what  has  been 
achieved.  But  in  this  second  case  we  are  referring  to  the 
type  of  contract  wherein  a  certain  milestone  must  be 
achieved.  As  an  example,  it  may  be  that  a  trainer,  meeting 
certain  miminum  capabilities,  must  be  produced  within  the 
time  frame  and  budget  limits.  If  unforeseen  technical  pro¬ 
blems  arise,  the  extra  effort  required  to  fulfill  the  terms 
of  the  contract  may  be  at  the  contractor's  own  expense. 

The  division  of  contracts  into  technology  oriented  and 
product  oriented  groupings  is  obviously  an  oversimplifica¬ 
tion  but  it  serves  a  descriptive  purpose.  Most  all  the 
6.1  efforts  and  some  6.2  efforts  are  of  course  technology 
oriented,  while  the  rest  of  the  6.2  efforts  and  the 
remaining  developmental  levels  are  product  oriented.  Since 
development  firms  and  research  organizations  have  tended  to 
specialize  in  terms  of  the  type  of  contracts  they  pursue,  a 
discontinuity  in  the  total  process  as  described  in  Figure  23 
has  resulted.  Thus  the  basic  researcher,  who  might  be 
working  on  a  knowledge  representation  system  (KRS)  for 
example,  selects  a  subject  domain  within  which  to  work,  and 
then  proceeds  until  his  ideas  are  proven  viable.  He  then 
leaves  that  particular  effort  to  tackle  other  basic 
research  problems . 

In  developing  a  training  system  such  as  the  automated- 
speech-based  systems,  the  product  oriented  contractor  may 
be  aware  of  the  previous  achievements  in  knowledge  repre¬ 
sentation  systems.  But  having  not  been  a  part  of  the  basic 
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research  effort,  his  personnel  may  not  have  the  technical 
experience  required  to  efficiently  implement  that  technol¬ 
ogy.  Further,  once  the  basic  research  contract  had  proven 
the  viability  of  the  KRS,  it  would  not  be  rational  to  ex¬ 
pect  the  basic  researcher  to  explore  the  details  of  all 
possible  applications.  Thus  when  the  KRS,  though  sound 
conceptually,  is  applied  specifically  to  one  of  the 
autonated-speech-based  systems,  problems  may  arise  requiring 
unanticipated  resources  to  resolve.  Knowing  this,  the  pro¬ 
duct  oriented  contractor  is  reluctant  to  attempt  to  imple¬ 
ment  new  technologies  when  he  is  responsible  for  the 
"result"  not  just  the  "process."  In  fact  he  may  be  reluctant 
to  bid  on  an  RFP  which  requires  implementation  of  a  KRS 
because  it  is  too  open-ended.  Being  unfamiliar  with  a  viable 
but  nonspecific  technology,  he  is  unable  to  estimate  the 
costs  of  the  RAD  which  must  precede  implementation. 

Awareness  then  is  only  part  of  the  problem.  The  gaps 
between  the  state  of  the  technology  when  it  leaves  the 
basic  researcher  and  the  state  which  would  be  required  by 
the  product  oriented  contractor  is  the  other  part.  One 
solution  would  be  for  the  applied  contractor  to  engage  in 
his  own  RAD  effort  to  fill  the  gap  so  that  he  might  stay 
competitive  when  bidding  on  automated-speech-based  systems. 
This  would  have  to  be  taken  from  the  profit  margin  and  am¬ 
ortized  over  several  development  efforts.  This  is  some¬ 
what  prohibitive  for  most  firms,  however,  in  that  the 
government  accounting  system  attempts  to  keep  the  profit 
rate  down  and  only  allows  direct  development  costs  to  be 
charged  against  a  particular  contract.  Thus  general  RAD, 
if  it  can  not  be  attributed  to  a  specific  contract,  must 
come  out  of  the  limited  profits.  In  interviews  with 
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personnel  from  the  Defense  Contracts  Accounting  Agency 
(dcaa)  ,  there  are  some  exceptions  to  our  statements,  wherein 
contractors  may  claim  the  expenses  of  general  R&D.  Some 
readers  may  want  to  explore  those  possibilities  further. 

Another  solution  is  for  the  government  to  attempt  to 
fill  the  technological  gaps.  This  could  be  done  by  a 
series  of  6.1  or  6.2  level  research  oriented  contracts  in 
which  the  goal  of  the  efforts  is  not  new  breakthroughs 
(as  in  the  basic  research  efforts),  but  rather  the  exten¬ 
sion  or  application  of  technologies  already  achieved  in 
the  basic  domain.  For  example,  an  '‘application  R&D” 
effort  could  be  designed  to  extend  present  developments 
in  a  KRS  to  the  knowledge  domain  of  an  AIC,  wherein  the 
further  extension  to  PAR  and  LSO  would  require  only  mini¬ 
mal  effort  *  An  illustrative  example  of  the  efforts  des¬ 
cribed  is  shown  in  Figure  2k.  It  would  be  beneficial  if 
the  "application  R&D"  effort  spanning  the  automated- 
speech-based  systems,  could  be  executed  by  some  of  the  same 
firms  that  might  be  developing  the  systems  at  the  lower 
levels.  This  would  provide  for  a  continuity  in  personnel. 
Otherwise,  the  efforts  could  be  carried  out  by  other  con¬ 
tractors  or  the  human  factors  laboratories  monitoring  the 
efforts.  Care  would  have  to  be  taken  in  that  case  however 
to  ensure  that  the  "extension"  or  "application"  technology 
gets  transmitted  to  the  "product  oriented"  contractors 
down  the  line. 

A.  final  possible  solution  is  to  feed  the  information 
as  to  the  needs  of  the  automated-speech-based  development 
back  to  some  of  the  basic  researchers.  Often,  when  devel¬ 
oping  training  technology,  the  basic  researcher  needs  a 
subject  domain  in  which  to  work.  In  fact,  several  have 
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Figure  24.  Proposed  Organizational  Structure 
of  R&D  Efforts 


shown  interest  in  the  GCA-CTS.  If  they  were  to  adopt  a 
GCA-CTS-like  medium  in  which  to  work,  little  effort  would 
subsequently  be  required  to  implement  the  system  in  a 
real  training  environment . 

SOLUTIONS.  Two  possible  solutions  present  themselves: 

(l)  tackling  the  awareness  problem  by  increasing  the  cross 
feed  of  information  between  the  basic  researcher  and  the 
product  oriented  contractor;  and  (2)  funding  "application 
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R&D"  efforts.  During  the  month  of  July  1979*  meetings  were 
held  with  representatives  from  ONR,  NAVTRAEQUIPCEN,  and  the 
senior  author  from  Behavioral  Evaluation  &  Training  Systems 
(BETS),  in  which  these  problems  were  discussed.  One  solu¬ 
tion  proposed  at  these  meetings  was  to  hold  two  conferences, 
approximately  a  year  apart.  At  the  first  conference, 
product  oriented  contractors,  and  representatives  of  the 
monitoring  human  factors  laboratories  would  present  the 
needs  of  current  and  future  training  systems,  similar  to 
tho  three  systems  described  in  the  present  report.  The 
audience  in  this  first  conference  would  be  a  group  of 
basic  researchers.  In  the  second  meeting,  the  roles  of 
speakers  and  audience  would  be  reversed.  Hie  basic  re¬ 
searchers,  having  been  exposed  to  applied  needs,  would 
present  their  technologies  and  explanations  as  to  what 
the  gaps  are  estimated  to  be,  and  the  kinds  of  efforts 
required  for  implementation.  At  the  time  of  this  writing, 
a  conference  is  being  planned. 

Other  possible  solutions  discussed  were  the  reorgani¬ 
zation  and  dissemination  of  summaries  of  ONR  efforts  and 
periodic  reports  generated  to  review  basic  technologies 
for  selected  applications.  Additionally  exploratory  6.2 
level  contracts  could  be  written  so  as  to  be  more  research 
oriented  rather  than  product  oriented  to  allow  the  systems— 
development  firms  to  engage  in  the  R&D  required  to  extend 
basic  technologies  thus  increasing  continuity.  The  firms 
themselves  could  also  explore  their  cost  accounting  prac¬ 
tices  with  the  DCAA  and  other  agencies  to  see  if  their  own 
R&D  could  be  amortized  over  projects. 
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SECTION  VI 
SUMMARY 

This  report  has  examined  the  training  characteristics 
of  the  Ground  Controlled  Approach  (GCA)  radar  controller, 
the  Landing  Signal  Officer  (LSO)  and  the  Air  Intercept 
Controller  (AIC).  The  expert  and  novice,  in  respect  to 
these  three  tasks  were  contrasted  in  terms  of  their  rela¬ 
tive  skills.  The  novice  was  characterized  as  a  controlled 
serial-processing  system,  unable  to  effectively  perform 
in  a  concurrent  task  environment,  wherein  the  tasks  com¬ 
pete  for  the  same  pool  of  central  resources.  The  expert 
on  the  other  hand,  was  characterized  as  a  resource  effi¬ 
cient  system  capable  of  preprocessing  and  automatic  res¬ 
ponding.  Several  areas  of  basic  research  were  reviewed 
in  search  of  a  set  of  technologies  with  which  an  instructor 
model,  resident  in  an  automated  speech- recognition-  based 
training  system,  might  be  designed.  Research  in  cognitive 
processing  limitations  and  components  were  examined  in 
some  depth  along  with  recent  developments  in  intelligent 
knowledge-based  computer  assisted  instruction  (CAl). 

The  result  was  an  outline  of  the  characteristics  of  a 
prototypic  instructor  model.  Some  of  the  technologies 
incorporated  in  the  prototype  were  insufficiently  developed 
along  the  lines  of  the  three  aircraft  controller  tasks  and 
thus  further  research  and  development  (R&D)  would  be  re¬ 
quired  for  implementation.  Finally,  recommendations  for 
R&D  and  the  dissemination  of  information  were  discussed. 

In  short,  this  report  represents  an  attempt  to  summarize 
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and  translate  some  of  the  achievements  of  basic  research 
into  their  relevance  for  the  continuing  development  of 
automated  speech-based  training  systems. 
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DESCRIPTIONS  OF  RECENT  CAI  RELATED  RESEARCH  BY  ONR. 


The  Office  of  Naval  Research  (ONR)  sponsors  two  main 
areas  of  research  in  order  to  enable  improvements  or  advances 
to  be  made  in  naval  operations.  The  first  area  involves 
basic  or  fundamental  research.  The  efforts  to  gain  funda¬ 
mental  knowledge  are  generally  of  long  term  and  are  aimed 
at  obtaining  information  leading  to  the  solutions  to  naval 
problems.  In  the  second  area  research  efforts  involve 
development  of  new  technologies  and  the  testing  of  new 
concepts  for  naval  operations.  Since  the  needs  of  naval 
operations  span  diverse  fields  of  specialization,  much 
of  the  research  sponsored  by  ONR  is  interdisciplinary. 

Because  of  the  broad  spectrum  of  factors  affecting 
naval  operations,  ONR  has  been  organized  into  ten  specific 
science  and  technology  divisions  to  handle  the  research 
needs  of  the  Navy.  The  divisions  range  from  the  Naval 
Vehicles  and  Weapons  Technology  Division  through  the 
Biological  Sciences  Divisions  and  the  Arctic  and  Earth 
Sciences  Division  to  the  Analysis  and  Support  Division. 

Of  the  ten  divisions,  the  Psychological  Sciences  Division 
is  organized  into  18  clusters  with  five  of  these  clusters 
pertinent  to  CAI:  information  processing  abilities, 
representation  of  knowledge,  cognitive  processes,  visual 
and  auditory  perception,  and  information  processing  and 
decision  making.  The  latter  two  clusters  are  relevant  in  so 
far  as  they  each  provide  information  about  the  basic  under¬ 
lying  functions  of  human  information  processing.  The  other 
three  clusters  are  more  directly  involved  in  the  development 
of  CAI. 

The  following  information  describing  the  clusters  and 
the  research  conducted  within  the  framework  of  the  clusters 
was  obtained  from  the  fiscal  year  programs  booklets  produced 
by  the  Psychological  Sciences  Division.  The  research  efforts 
cited  here  should  not  be  considered  as  reviews  but  descriptions 
illustrating  the  research  sponsored  by  the  Psychological 
Sciences  Division  and  ONR. 

The  first  cluster  to  be  discussed  deals  with  information 
processing  abilities.  The  purpose  of  research  conducted  in 
this  cluster  is  to  investigate  basic  information  processing 
operations  that  underlie  task  abilities .  There  are  three 
major  areas  of  interest  within  the  information  processing 
abilities  cluster:  the  implication  of  individual  differences 
in  the  parameters  of  some  basic  information  processing  func¬ 
tions,  performance  in  complex  tasks  as  a  function  of  profi¬ 
ciency  in  the  basic  information  processing  functions  under¬ 
lying  those  tasks,  and  the  potential  of  information  processing 
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theories  to  provide  substantive  explanations  of  the  relation¬ 
ship  between  mental  test  performance  and  performance  in  real 
world  tasks. 

The  first  work  unit  to  be  described  is  one  by  J.B.  Carroll. 
The  first  of  three  proposed  reports  has  already  been  completed. 
The  report  consists  of  two  parts:  a  critique  of  correlational 
techniques  and  related  statistics  as  applied  to  information 
processing  research  on  individual  differences,  and  an 
analysis  concerning  the  conclusions  to  be  drawn  about  traits, 
factors,  cognitive  processes,  and  individual  differences. 

In  the  next  work  unit  begun  in  FY77,  a  test  battery 
designed  for  use  as  an  assessment  instrument  for  the  evalua¬ 
tion  of  performance  is  being  developed.  A.M.  Rose  in  an 
initial  version  of  the  battery  has  included  such  tasks  as 
mental  comparison,  memory  scanning,  and  graphemic  and  phonemic 
classification.  Some  other  tasks  which  may  be  included  in  the 
final  battery  include  free  recall,  susceptibility  to  inter¬ 
ference,  and  mental  calculation. 

R.J.  Sternberg  began  a  planned  five-year  effort  in  FY78 
with  the  formulation  and  evaluation  of  a  general  theory  of 
human  reasoning  components.  His  theory  assumes  deductive 
and  inductive  reasoning  processes  are  the  components  under 
study.  According  to  the  theory,  there  may  be  some  individual 
variation  in  the  use  or  non-use  of  the  component  processes 
forming  the  mental  representations  upon  which  the  processes 
act.  In  addition  individuals  may  also  vary  according  to  the 
strategies  employed  in  the  blending  of  the  processes,  how 
consistantly  those  strategies  are  applied,  and  in  the  speed 
and  accuracy  of  the  component  processes.  Sternberg  is  also 
formulating  specific  mathematical  models  of  simple  reasoning 
tasks  involving  the  component  processes.  These  mathematical 
models  could  then  be  used  in  characterizing  individual  dif¬ 
ferences  and  relating  the  differences  to  standard  psycho¬ 
metric  measure  of  related  abilities. 

B.  Schneider  in  some  of  his  earlier  work  formulated  a 
theory  that  classified  detection  tasks  as  either  control 
processing  or  automatic  processing.  In  control  processing, 
tasks  are  accomplished  by  a  slow,  serial,  capacity-limited 
process  having  a  considerable  cognitive  load.  Automatic 
processing,  on  the  other  hand,  occurs  over  a  fast,  parallel 
capacity,  unlimited  channel  with  little  if  any  cognitive 
load.  The  automatic  processing  becomes  possible  only  after 
extensive  training  with  a  consistent  stimulus-response  map¬ 
ping.  The  project  funded  by  ONR  which  began  during  FY78 
will  look  as  the  implications  of  Schneider’s  theory  in 
individual  differences  and  the  prediction  of  performance. 
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In  a  one  year  project  begun  FY78,  J.E.  Hoffman  is 
investigating  the  acquisition  of  automatic  detection  by  a 
fast,  unlimited  capacity  mechanism  requiring  little,  if  any 
cognitive  effort.  The  output  of  this  mechanism  is  described 
as  an  index  of  the  similarity  of  the  stimulus  and  target  sets. 
Medial  values  of  the  index  produce  a  slow,  serial  process, 
while  values  at  either  extreme  permit  the  subject  to  make 
immediate  responses.  Through  training  a  subject  refines 
the  similarity  index: toward  the  extremes.  The  goal  of 
Hoffman's  project  is  the  formulation  of  a  model  allowing 
quantitative  evaluation  of  the  effects  of  individual  dif¬ 
ferences  during  training  upon  the  similarity  index. 

A  research  effort  begun  in  TY77  is  focusing  on  the 
hypothesis  that  "complex  tasks  draw  upon  a  reservoir  of  spare 
capacity"  which  varies  between  individuals  in  its  size.  By 
measuring  the  performance  degradation  upon  a  concurrent 
secondary  task,  E.  Hunt  has  been  able  to  determine  the  task 
demands  of  a  given  primary  task  on  the  capacity  reservoir. 
Using  this  measurement  technique,  Hunt  is  attempting  to 
formulate  a  general  method  which  will  allow  performance  on 
complex  tasks  to  be  predicted  by  measuring  performance 
requirements  of  the  simpler  component  tasks.  Prediction  of 
performance  in  difficult  problem-solving  situations  could 
be  accomplished  without  exposing  personnel  to  the  target 
situation  by  applying  the  methodology  being  prepared  by  Hunt. 

S.W.  Keele  was  attempting  to  obtain  information  which 
would  allow  some  resolution  to  the  conflicts  between  various 
theories  of  attention,  particularly  as  they  relate  to 
personnel  selection  and  individualization  of  training.  The 
approach  taken  in  this  three  year  effort,  completed  in  FY78, 
involved  the  flexibility  of  attention.  Individuals  differ  in 
terms  of  the  point  where  they  no  longer  incorporate  items 
and  begin  to  exclude  items  of  information  from  attention. 
During  the  course  of  this  project,  Keele  found  the  "fusion" 
phenomena  (the  fusing  of  dichotic  inputs  such  as  "lanet"  and 
^panet"  into  a  single  word  "planet")  to  be  more  a  function  of 
subjects  having  difficulty  in  determining  the  temporal  order. 
He  also  found  that  individual  differences  in  memory  span  are 
not  attributable  to  variation  of  memory  strategy.  In  a  third 
problem  area  investigated  by  Keele,  a  theory  of  attention  was 
developed  placing  capacity  limits  on  response  selection  rather 
than  stimulus  analysis. 

The  result  of  a  planned  three  year  effort  begun  in 
FY78  by  Keele  is  a  test  battery  capable  of  predicting  complex 
task  performance  based  upon  simple  tasks.  The  project  started 
by  obtaining  pure  measures  of  five  hypothesized  cognitive 
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ability  factors.  Concurrent  validity  studies  of  the  battery 
will  be  performed  once  measures  have  been  established  for  all 
the  cognitive  ability  factors.  The  final  phase  of  the  project 
is  to  conduct  predictive  validity  studies  using  flight- school 
students . 

J.R.  Frederiksen  began  a  three-year  effort  in  TY77 
aimed  at  the  identification  of  perceptual  and  cognitive  skills 
related  to  reading  and  verbal  ability.  In  the  course  of  the 
research,  techniques  for  measuring  these  skills  will  be 
formulated  and  the  effects  of  skill  deficiencies  on  reading 
ability  will  be  investigated.  A  series  of  experiments  have 
defined  skill  deficiencies  of  poor  readers  and  five  basic 
components  of  reading  have  been  determined. 

The  ability  to  process  information  at  the  appropriate 
level  also  seems  to  vary  across  individuals.  In  this  regard, 
two  types  of  individuals  have  been  classified^  language  bound 
and  language  optional.  Language  bound  people  are  constrained 
in  their  thinking  by  the  rules  of  their  language  or  other 
high-level  conceptual  schemata,  while  language  optional  people 
are  able  to  operate  without  the  language  constraints  when 
the  task  requires  this  of  them.  R.S.  Day  in  this  three  year 
effort  begun  in  FY77  has  found  that  perceptual  or  sensory 
deficits,  intelligence,  or  other  artifacts  are  not  the  source 
of  the  phenomenon.  Additionally,  language  bound  people  have 
been  found  to  process  stimuli  to  deeper  levels  of  analysis. 
During  the  final  phase  of  the  project,  data  was  to  be  obtained 
on  demographic  and  biographic  factors. 

A  research  project  by  R.E.  Snow  begun  in  FY77  was 
seeking  to  understand  situations  in  which  aptitude-treatment 
interaction  occur.  He  constructed  an  aptitude  theory  which 
allows  the  abilities  measured  by  aptitude  test  to  be  repre¬ 
sented  in  terms  of  the  information  processing  required  by 
the  tests.  The  theory  goes  on  to  relate  the  information 
processing  to  the  interaction  of  instructional  and  differing 
individual  aptitudes.  An  information  processing  description 
of  the  aptitude  treatment  interaction  is  possible  with  a 
theory  such  as  this.  Data  indicates  a  two-step  process  is 
followed  by  individuals  who  are  successful,  high-aptitude 
processors.  The  process  consists  of  constructim the  eorrect 
answer  and  searching  the  response  alternatives  for  the  answer. 
A  less  systematic,  parallel  processing  of  both  items  and 
responses  characterizes  the  less  successful,  low  aptitude 
processors.  Indications  are  that  an  individual's  strategic 
flexibility  may  be  more  important  in  determining  aptitude 
affecting  learning  than  had  been  suspected. 
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E.A.  Fleishman  in  an  earlier  work  used  factor  analytic 
techniques  to  analyze  job  performance  and  suggested  that  37 
abilities  can  be  used  to  describe  that  performance.  Following 
up  on  this  finding,  Fleishman  began  an  investigation  into  the 
general  trainability  of  these  abilities.  The  hypothesis  is 
that  certain  procedures  used  in  the  training  of  specific 
abilities  will  allow  a  high  degree  of  transfer  to  all  tasks 
making  use  of  those  abilites.  Three  abilities  have  been 
chosen  for  training  and  the  experimental  designs  developed 
to  test  for  the  breadth  of  training  generalization.  "Valida¬ 
tion  of  this  training  transfer  concept  was  planned  for  the 
last  year  of  funding. 

A  theory  of  memory  developed  in  1969  by  B.J.  Underwood 
which  posits  that  memories  are  a  collection  of  attributes 
forms  the  basis  for  a  three  year  effort  completed  during 
FY78.  Using  a  factor  analytic  technique  on  a  variety  of 
tasks,  Underwood  identified  five  factors.  These  factors 
however  related  more  to  the  form  of  the  stimuli  and  task 
requirements  than  to  memory  attributes.  This  finding  was  of 
substantial  theoretical  interest  since  all  tasks  studied 
relied  heavily  on  associative  processes  leading  to  an  inter¬ 
pretation  that  associative  processes  may  be  multidimensional. 
The  results  from  another  thrust  of  the  work  suggest  that 
simultaneous  as  opposed  to  successive  learning  of  tasks 
enhances  long  term  retention. 

Observing  that  most  studies  of  human  memory  involve  a 
single  task  while  real-world  training  requires  an  individual 
to  learn  a  variety  of  tasks  and  switch  among  them  on  an 
irregular  basis,  Underwood  began  a  new  research  effort  in 
FY78  to  investigate  learning  in  multiple  task  situations. 

The  main  concern  is  the  determination  of  the  gjneralizability 
of  known  laws  of  human  memory  to  a  situation  where  individuals 
must  learn  multiple  tasks  simultaneously. 

Given  the  marginal  reading  skills  of  new  recruits 
entering  the  service,  an  effort  to  determine  the  most 
efficient  remedial  program  would  seem  appropriate.  Carver, 
id  a  work  unit  completed  in  FY78,  attempted  to  relate 
reasoning  ability  to  reading  proficiency.  A  positive  rela¬ 
tionship  would  indicate  those  individuals  having  good 
reasoning  ability  would  gain  the  most  from  a  remedial  reading 
program.  However,  the  testing  of  students  previously  divided 
according  to  high  and  low  scores  on  a  test  of  reasoning 
ability,  following  extensive  reading  tutorials  administered 
by  computer,  Indicated  no  significant  differences  between 
group 8 . 


191 


NAVTRAEQUIPCEN  79-C-0085”1 


The  next  cluster  involves  representing  knowledge  for 
training.  The  psychological  foundations  or  computer  based, 
interactive,  adaptive  instruction  is  the  main  concern  of 
this  cluster.  There  are  two  areas  of  emphasis  in  this 
cluster:  the  representation  of  knowledge  and  the  process  of 
instruction  itself.  Knowing  how  to  represent  knowledge  is 
important  if  the  computer  in  a  CAI  program  is  to  function 
effectively.  The  computer  must  be  able  to  answer  natural- 
language  questions,  be  able  to  generate  examples  and  problems 
appropriate  to  the  learning  context,  and  evaluate  the 
student's  knowledge  of  the  subject  material  being  taught. 
Studying  the  process  of  instruction  will  allow  the  most 
effective  strategies  to  be  developed. 

Students  arriving  for  training  have  different  cultural 
and  educational  backgrounds  which  leads  to  a  wide  variance 
concerning  the  initial  knowledge  of  the  subject  matter  and 
their  motivation.  A  training  program  which  is  based  upon 
network  models  of  human  memory  would  allow  an  optimum  match 
of  the  instructional  material  to  the  student's  abilities. 

D.A.  Norman  in  a  project  completed  in  FY76  attempted  to 
build  such  network  models  and  to  construct  a  representation 
of  the  semantic  and  syntactic  structures  of  the  information 
to  be  learned  by  the  studert  in  order  to  build  an  optimizing 
training  program.  The  results  of  developing  such  a  program 
would  also  lead  to  new  understanding  of  human  memory. 

Continuing  his  research  into  semantic  network  repre¬ 
sentations,  Norman  started  a  new  project  in  FY77.  This 
project  has  two  focal  points,  the  investigation  of  basic 
principles  of  instruction  and  the  development  of  operational 
computer  based  tutorial  systems.  These  computer  systems  will 
be  used  as  testbeds  for  the  study  of  the  instruction  principles 
and  as  prototypes  for  potential  application.  A  completely 
automated  tutorial  system  which  uses  a  semantic  memory 
representation  of  the  subject  matter  is  being  explored. 

Norman  is  also  exploring  a  semi-automated  tutor  making  use 
of  a  state-dependent  teaching  strategy.  The  theoretical 
work  has  centered  on  two  aspects  of  learning:  that  learning 
is  not  a  homogenous  process  and  that  a  student's  prior  know¬ 
ledge  is  critical  in  acquiring  new  information. 

J.W.  Rigney  has  been  investigating  theoretical  and 
empirical  bases  of  instruction  so  as  to  improve  the  effec¬ 
tiveness  of  training.  The  focus  of  the  empirical  investi¬ 
gation  has  been  the  development  of  a  computerized  hands-on 
maintenance  training  testbed  and  the  advanced  computer 
techniques  required  for  such  development.  A  computer-based 
General  Maintenance  Training  System  (GMTS)  has  been  developed 
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and  field  tested.  Further  real-world  testing  of  the  GMTS 
is  underway.  The  theoretical  emphasis  has  been  the  devel¬ 
opment  of  a  general  theory  of  computer-based  instruction. 
Studies  completed  have  involved:  descriptions  of  cognitive 
processing  in  terms  of  schema  theory,  strategies  used  in 
the  extracting  of  logical  relationships  from  written  mat¬ 
erial,  electrophysiological  correlates  of  cognitive  pro¬ 
cessing,  and  the  effect  on  summaries  and  recall  of  different 
text  structures. 

In  a  related  effort  begun  FY77,  Rigney  focused  on  the 
formulation  of  a  model  to  teach  the  self- directional  pro¬ 
cesses  involved  in  on-the-job  training  using  technical  mat¬ 
erial.  This  model  would  Be  used  by  military  personnel  as 
an  aid  in  their  own  self-directed  training.  This  would  be 
accomplished  by  teaching  them  how  to  select  their  goals  and 
then  obtain  the  relevant  information  from  the  technical 
documents  used  for  the  self-directed  training. 

Traditional  CAI  systems  are  ’’frame  oriented”  and  make 
use  of  "canned”  questions  and  the  answers  to  these  questions 
must  be  one  of  those  "canned"  answers  anticipated  by  the 
human  instructor  at  the  time  of  programming.  As  a  result, 
these  traditional  CAI  systems  have  limited  flexibility  and 
a  restricted  potential  for  the  individualizing  of  instruction. 
A.  Collins  expanded  an  existing  "mixed  initiative"  CAI  system 
SCHOLAR,  which  is  capable  of  generating  its  own  questions  and 
making  appropriate  responses  to  inputs  from  students.  Such 
a  system  uses  a  self-contained  hierarchically  structured 
data  base  of  facts,  concepts,  and  procedures.  In  this  re¬ 
search,  Collins  sought  the  improvement  of  the  interactive 
capabilities  of  a  CAI  system  through  the  development  of 
programs  and  techniques  allowing  continuous  natural- 
language  dialogue . 

In  a  subsequent  effort  started  during  FY77,  Collins  is 
attempting  to  use  a  generative  CAI  system  to  teach  causal 
knowledge  and  reasoning.  Using  this  system,  various  tutoring 
strategies  can  be  tested  for  the  relative  effectiveness  while 
holding  all  other  variables  constant,  something  not  possible 
with  a  human  tutor.  Additionally,  a  theory  of  tutoring 
developed  through  analysis  of  tutor-student  dialogues  is  being 
Incorporated  into  the  CAI  system  to  enable  the  program  to 
determine  the  student's  current  status  and  make  a  diagnosis 
of  errors.  The  optimal  strategy  will  be  determined  by  com¬ 
paring  the  learning  levels  attained  by  the  students  under 
different  versions  of  the  program. 

J.R.  Anderson  is  developing  a  computer  model  for  the 
learning  of  skills  and  procedures  as  opposed  to  the  learning 
of  facts.  The  resulting  computer  programs  are  to  be  "taught" 
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from  topic  areas  and  the  output  of  the  simulation  examined 
for  general  aspects  of  human  learning.  Use  of  this  model 
should  enable  the  evaluation  of  the  effects  of  -various 
training  techniques,  the  teaching  of  new  training  skills, 
and  the  evaluation  and  remediation  of  cognitive  skills 
deficiencies.  One  of  the  results  already  obtained  suggests 
principles  of  learning  acquired  from  s tudies-of- fact  learning 
may  be  more  generalizable  to  the  learning  of  procedures  than 
expected. 

A  one-year  effort  by  K.T.  Wescourt  completed  during 
FY78  examined  the  possible  improvement  of  students'  abilities 
to  solve  troubleshooting  (debugging)  problems  if  they  are 
given  explicit  instruction  in  general  problem-solving  methods 
rather  than  the  students  acquiring  the  methods  through  triAl- 
and-error  learning.  Data  analysis  indicated  that  experienced 
programmers  debugging  expertise  is  the  result  of  their  idio¬ 
syncratic  experience  and  the  differences  in  the  debugging 
competence  of  inexperienced  programmers  is  a  function  of 
their  use  of  general  problem-solving  methods.  Failure  to 
adequately  diagnose  a  program  malfunction  and  an  inability 
to  back  up  when  an  attempted  correction  failed  as  a  cure  are 
the  characteristics  frequently  associated  with  the  inefficient 
and  unsuccessful  debugging  attempts  of  inexperienced  pro- 
trammers  . 

P.  Suppes  and  K.T.  Wescourt  completed  a  research  effort 
during  FY77  dealing  with  a  generative  CAI  system  which  made 
use  of  a  network  representation  of  the  course  content  to  be 
taught.  Individualized  teaching  was  made  possible  through 
the  use  of  a  "curriculum  driver"  which  developed  a  sequence 
of  tasks  for  each  student  based  upon  the  contents  of  the 
network  and  made  reference  to  the  current  status  of  the 
student's  mastery  of  the  target  skills. 

Previously  undiscovered  conventions  that  regulate  human 
dialogue  were  identified  by  W.C.  Mann  in  a  study  completed 
during  FY77.  Termed  "Dialogue -Games,"  these  conventions 
are  recurrent  structures  defined  by  the  goals  and  subgoals 
of  the  dialogue.  Mann  used  naturally  occuring  human 
dialogues  as  the  source  of  his  data.  A  Dialogue-Game  was 
found  to  communicate  a  large  number  of  non-explicit  under¬ 
standings.  Much  of  the  brevity,  interpretive  selectivity, 
and  covert  inference  found  in  ordinary  human  interaction  is 
attributable  to  these  Dialogue-Games.  It  is  suggested  that 
man-machine  interaction  could  be  more  concise  and  more 
flexible  if  these  Dialogue-Games  could  be  transferred  to 
man-machine  systems. 

The  last  cluster  of  the  Psychological  Sciences  Divi¬ 
sion  to  be  described  involves  cognitive  processes.  This 
was  a  program  initiated  in  FY78  to  characterize  skilled 
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performance  in  real  world  tasks  demanding  complex  informa¬ 
tion  processing.  The  cognitive  processes  program  emphasizes 
four  main  areas;  information  collection  and  analysis,  the 
representation  of  task  domains  by  problem  solvers  of  varying 
levels  of  expertise,  search  and  planning  processes  involved 
in  skilled  problem  solving,  and  important  features  in  the 
execution  of  problem  solutions. 

D.  Kieras  is  studying  aspects  of  text  comprehension  in 
an  effort  to  discover  how  people  determine  the  subject  of 
reading  material  after  only  processing  superficial  features 
of  the  text  and  the  full  comprehension  or  knowledge  of  the 
relevant  context  and  background  has  yet  to  be  attained.  This 
research  has  value  in  that  jobs  involving  text  comprehension 
generally  require  the  reader  to  decide  the  topic  or  subject 
of  the 'material  iir.order  to  decide  relevancy  of  the  material 
and  obtain  the  overall  context  required  for  deeper  concen¬ 
tration.  Results  from  experiments  will  be  used  to  £orm  a 
computer  simulation  model  which  is  to  be  checked  against 
behavior  data.  The  simulation  would  then  be  applied  to 
describe  topic  identification  strategies  and  as  an  aid  to 
the  training  of  more  efficient  topic  identification. 

W.  Kintsch  has  developed  a  theory  of  text  compre¬ 
hension  which  allows  the  evaluation  of  a  readers'  profi¬ 
ciency  in  decomposing  text  into  a  set  of  propositions  repre¬ 
senting  its  meaning.  These  propositions  become  the  input 
of  a  concept  learning  theory  by  L.  Bourne  that  describes 
how  people  determine  which  types  of  information  predict  the 
value  of  an  item  or  task  based  upon  their  experience.  This 
joint  effort  by  Bourne  and  Kintsch  will  examine  the  effect 
of  different  text  structures  on  the  prediction  task  and  the 
parameters  of  the  concept  learning  of  the  task. 

J.G.  Greeno  is  concerned  with  the  relationship  of  the 
syntactic  properties  to  the  semantic  properties  of  a  domain. 
Choosing  plane  geometry  as  a  prototypical  case  because  the 
axioms  and  theorems  contain  syntactic  properties  and  the 
semantic  properties  can  be  represented  by  diagrams  and  other 

feometric  properties  of  construction,  Greeno  will  attempt  to 
ormulate  a  theory  of  geometry  problem  solving.  This  theory 
will  describe  the  roJ.es  and  interactions  of  axiomatic  formal 
reasoning  and  diagramatic  informal  reasoning.  In  a  related 
theoretical  work,  an  attempt  to  describe  the  causal  structure 
of  semantic  knowledge  provided  by  studies  of  rapid  scanning 
of  diagrams  and  other  properties  of  diagrams  in  a  semantic 
network  will  be  made. 

A  five-year  effort  to  study  the  differences  between 
novice  and  expert  problem  solvers  is  planned  by  R.  Glaser. 
This  analysis  will  use  two  classes  of  problems,  analytical 
and  perceptual,  to  determine  how  the  novices  differ  from  the 
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experts  in  their  representation  of  the  problems.  The  ability 
to  produce  relevant  facts  to  particular  problems  and  the 
effects  of  various  types  of  priming  on  the  appropriate 
representation  will  be  studied.  Also  to  be  studied  is  the 
ability  to  construct  and  draw  inferences  from  external 
representations.  Later  phases  of  the  project  will  investi¬ 
gate  such  aspects  of  problem  solving  as  planning,  search, 
execution,  and  checking.  Perceptual  fluency  will  be  combined 
with  artificial  intelligence  approaches  to  spatial  and 
locational  cognition  in  studies  of  perceptual  problem  solving. 
A  series  of  studies  allowing  an  analysis  of  the  patterns  and 
procedures  used  by  people  in  the  interpretation  of  maps  is 
also  planned. 

A  three-year  study  by  P.  Thom  dyke  addresses  four  main 
issues:  map  learning,  mental  map  construction,  representation 
of  geographical  information,  and  distance  estimation.  The 
analysis  of  an  initial  study  of  map  learning  indicates  indivi¬ 
dual  differences  may  be  attributable  to  Strategic  sources. 

From  work  in  the  area  of  mental  map  construction  it  was  found 
that  actual  navigational  experience  produces  more  locale 
knowledge  while  map  experience  enhances  the  acquisition  of 
global  knowledge.  A  dual-code  theory  which  assumes  both 
relational  and  analog  codes  for  spatial  knowledge  is  the 
central  theme  for  studies  of  the  representational  knowledge. 
The  effect  of  clutter  along  the  distance  to  be  estimated 
is  the  concern  of  distance  estimation  studies.  Clutter  seems 
to  cause  overestimation  of  distances,  apparently  a  phenomena 
unique  to  memorized  maps. 

A  theory  of  route  planning  is  the  topic  of  a  planned 
three  year  effort  by  A.  Stevens.  Taking  the  form  of  a 
"planning  grammar,"  the  theory  will  represent  various  types 
of  route  plans  and  describe  how  various  types  of  representa¬ 
tion  of  locale  are  used  by  people.  A  computer-based  simula¬ 
tion  will  be  used  to  empirically  evaluate  the  theory.  One  of  tie 
data  analysis  approaches  will  attempt  to  interpret  planning 
errors  in  terms  of  the  plans  or  subplans  which  produce  the 
errors . 

A  theory  of  human  planning  in  the  context  of  software 
design  is  the  goal  of  a  project  headed  by  M.  Atwood.  Planned 
for  two  years,  the  effort  will  investigate  the  theoretical 
assumption  that  planning  takes  the  form  of  a  "procedural 
network."  The  internal  assumption  of  the  procedural  network 
is  that  planning  occurs  by  expanding  each  abstract  part  into 
more  concrete  subparts  and  by  ensuring  the  appropriate 
coordination  of  the  generated  subparts.  The  techniques 
developed  in  the  project  will  be  used  in  a  variety  of  ways, 
for  example:  the  order  of  expansion  of  the  networks  can  be 
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determined  and  an  examination  of  the  ways  in  which  design  and 
planning  knowledge  are  combined  with  planning  of  the  tasks  or 
problem  domain  can  be  performed. 

B.  Hayes -Roth  plans  to  use  an  "opportunistic"  theoretical 
approach  to  human  planning  rather  than  a  "hierarchical" 
approach  as  used  by  Atwood.  This  theory  holds  that  a  number 
of  parallel,  independent  knowledge  sources  accomplish  the 
planning  by  communicating  through  a  central  data  base  which 
records  the  plan  and  other  related  material.  Each  of  these 
knowledge  sources  is  capable  of  specific  modification  of  the 
data  base  whenever  the  required  conditions  are  met.  Hier¬ 
archical  planning  of  the  type  described  by  Atwood  is  possible 
since  some  of  the  knowledge  sources  act  on  abstract  data  by 
expanding  the  data  into  more  concrete  forms.  However,  this 
form  of  hierarchy  can  flow  in  the  reverse  direction  with  some 
knowledge  sources  creating  abstract  data  from  the  more 
concrete  data  presented  as  input. ' 

The  Mathematic  Sciences  division  of  ONR  (as  described 
in  the  ONR  Publication:  Contract  Research  and  Technology 

Program)  also  sponsors  research  relevant  to  CAI.  This 
di>  .sion  has  three  main  areas:  operations  research,  statis¬ 
tics  and  probability,  and  information  systems.  The  research 
programs  sponsored  in  this  last  area  are  the  ones  of  interest 
in  the  development  of  CAI.  The  information  systems  area  has 
four  program  thrusts:  computer  hardware  and  device  design, 
large  file  systems,  software  design,  and  man-machine  symbiosis. 
Of  these  four  programs,  the  study  of  man-machine  symbiosis 
most  directly  applies  to  CAI.  7our  program  areas  comprise 
the  man-machine  symbiosis  area:  computer-aided  design,  arti¬ 
ficial  intelligence,  natural  language  interaction,  and  speech 
understanding.  Only  the  last  three  areas  apply  to  CAI 
development.  The  following  research  descriptions  are  the 
result  of  personal  communication  with  Gordon  Goldstein, 
Director  of  Information  Systems  Programs. 

The  application  of  artificial  intelligence  (AI)  to  a 
CAI  system  would  allow  that  system  to  become  more  dynamic 
and  flexible  in  its  interaction  with  students.  Some  of  the 
AI  research  sponsored  by  the  Mathematical  Sciences  Division 
is  allowing  the  development  of  AI  techniques  which  could  be 
applied  to  CAI  systems. 

A  work  unit  being  conducted  by  C.  Hewitt,  involves  the 
application  of  AI  to  the  solving  of  large  data  base  problems. 
Although  aimed  at  problems  in  the  logistics  environment,  CAI 
systems  also  will  nave  difficulty  with  large  files  used  for 
:ne  storage  of  the  accumulated  knowledge  needed  for  a 
dynamic,  flexible  CAI  system. 
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R.  Reddy  is  investigating  the  development  of  AI  in  the 
direction  of  dynamic  acquisition  of  world  knowledge  by  the 
computer  and  the  ability  of  the  computer  to  modify  its 
behavior  according  to  percieved  changes  in  the  external 
environment.  In  addition,  work  has  involved  the  development 
of  automatic  visual  learning  through  the  combination  of 
symbolic  and  signal  processing  information.  Addition  of 
features  such  as  these  would  allow  the  human  teacher  or  tutor 
to  give  the  CAI  system  the  knowledge  required  for  teaching 
and  to  usual  visually  presented  material  in  that  teaching. 

The  ability  to  modify  its  behavior  according  to  input  from 
the  environment  would  aid  in  the  CAI  systems'  capability  to 
adapt  to  changes  in  student  performance  more  readily. 

A  project  directed  by  E.  Chamiak  has  as  its  goal  the 
development  of  a  computer  system  capable  of  "reading''  a 
text  and  extracting  enough  information  to  allow  the  questions 
at  the  end  of  the  chapter  to  be  answered.  Thus  the  CAI 
system  could  "learn"  the  contents  of  a  course  text  to  form 
the  knowledge  base  used  in  the  teaching  of  that  course. 
Initially  a  program  to  solve  the  chapter  problems  given  a 
preprogrammed  knowledge  base  is  being  attempted.  Later 
phases  will  be  concerned  with  the  acquisition  of  knowledge 
by  the  system. 

M.  Minsky  is  seeking  to  establish  a  theory  and  process 
of  "knowledge  based  computer  problem  solving."  His  approach 
is  to  view  an  intelligent  problem  solver  as  a  network  or 
society  of  specialists  with  each  specialist  having  a  large 
data  base  applicable  to  its  domain.  These  specialist 
computer  system  components  are  each  capable  of  solving 
problems  within  its  domain.  Three  possibilities  are  pro¬ 
posed  for  the  integration  of  the  specialists  in  the  solving 
of  a  general  problem:  an  expert  on  the  specialists  to 
distribute  those  portions  of  the  problem  to  appropriate 
components,  a  negotiation  system  which  will  allow  the 
specialists  to  operate  in  parallel  upon  those  problem  com¬ 
ponents  having  the  greatest  association  with  their  exper¬ 
tise,  and  a  combination  of  expert  and  negotiation  systems. 

One  of  the  large  barriers  to  widespread  use  of  CAI 
systems  is  the  inability  of  the  system  to  communicate  with 
the  student  in  his  natural  language  rather  than  a  highly 
specialized  subset  of  the  language  having  very  limited 
vocabulary.  L.  R,  Harris  and  S.  J.  Garland  are  attempting 
to  improve  the  natural  language  capabilities  of  computers 
input  and  control.  In  the  same  vein,  D.L.  Waltz  is  attempting 
to  develop  a  language  system  which  will  selectively  retrieve 
information  from  data  storage,  as  a  response  to  user 
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interrogation,  and  respond  with  the  answer  in  readable  form, 
all  in  real-time.  The  goal  of  a  project  hedded  by  W.A.  Woods 
is  to  build  an  AI  system  capable  of  not  only  interacting  in 
natural  language  but  also  able  to  make  comparisons  and  draw 
analogies  over  narrative  data  or  to  generalize  over  informa¬ 
tion  sources.  The  resulting  products  of  these  research 
efforts  when  applied  to  CAI  systems  will  allow  non-computer 
oriented  personnel  to  interact  effectively  with  the  system. 

CAI  systems  could  then  become  even  more  generalized  in  their 
usage  and  more  flexible  in  their  interaction  v,;.th  students. 

Speech  decoding  and  synthesis  is  another  area  of 
research  which  is  aimed  at  more  natural  interaction  with 
computers.  J.D.  Markel  has  been  investigating  the  ability 
to  use  speech  synthesis  over  telephone  systems  and  has  met 
with  some  success.  Using  basic  research  in  speech  analysis 
and  synthesis  techniques,  J.E.  Shoup  is  attempting  to 
develop  the  means  ox  obtaining  automatic  derivation  of 
accoustical  parameters  for  automatic  recognition  and  synthesis. 
With  such  capabilities,  the  normalization  of  inter-speaker 
differences  becomes  more  of  a  reality.  In  a  real-world 
situation  a  student  may  seldom  have  access  to  a  CAI  system 
without  also  having  background  noise  levels  degrading  his 
speech.  A.V.  Oppenheim  has  been  working  toward  a  solution 
to  such  a  problem  by  developing  enhancement  techniques  for 
degraded  speech.  Research  such  as  described  in  these 
examples  will  provide  the  basis  for  techniques  allowing 
student-computer  interaction  at  a  voice  level  which  would 
more  closely  resemble  the  familiar  student-human  teacher 
interaction. 

Given  the  above  brief  discussion,  the  conclusion 
that  techniques  allowing  verbal  interchange  in  normal 
conversation  language  can  be  developed  seems  unavoidable. 
However,  much  research  and  development  in  this  area  is 
required.  Manual,  typed,  interaction  is  already  possible 
albeit  in  a  subset  of  the  natural  language.  Also  the  use 
of  AI  techniques  will  allow  more  flexibility  in  the 
student -computerized  tutor  interaction. 
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APPENDIX  B 

DISCUSSION  OUTLINE  IVN  ONE  MEETING 


Technical  Requirements 
Instructor  Model  for  Automated  Speech 
Recognition  Based  Training  Systems 


I.  Instructor  Tasks 

A.  Design  of  Curriculum:  Instructional  decisions  re¬ 

garding  curricular  design  made  off-line. 

B.  Dynamic  Decisions  during  Instruction:  On-line  deci¬ 

sions  regarding  instructional  alternatives. 

II.  Off-line  Design 

A.  Overall  structure  of  Curriculum 

1.  Macro  sequencing  of  concepts 

2.  Micro  schemata  to  facilitate  subsequent  performance 

B.  Design  of  Manual 

C.  System  Control  Questions 

1 .  Display  questions — 

2.  Choice  of  performance  measures 

3.  Feedback  techniques 

4.  Other  miscellaneous  training  system  questions 

III.  Dynamic  Instructional  Decisions 

A.  Establishing  schemata  (particularly  response  selec¬ 
tion  and  production. 

1 .  Generative  CAI 

a.  generation  of  frames  (rules),  scenarios  (eg.) 

b.  CIN  based 

2.  Performance  measures 

a.  success  counters  for  student  model 

b.  RT  measures  for  speeded  responses  on  paced 
skills 

c.  diagnostic  usage  of  errors  and  pauses 

3.  Feedback 

4.  Remediation 

5.  Optimizing  algorithms 
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B.  Establishing  Paced  Skills  (Adaptive  Training) 

1.  Break  skills  into  componants:  Move  one  componant 

from  controlled  to  auto  processing  before  working 
on  next  componant?  Because  of  limited  resources-- 
performance  breaks  down  in  early  controlled  pro¬ 
cessing 

2.  Adaptive  Variable (s):  Primarily  rate? 

3.  Student  Model 

4*  Performance  Measurement:  Latency  measures;  RT  on 
targets,  RT  on  probes.  Error  measures;  errors  on 
distractors  and  foils.  Performance  on  secondary 
tasks  as  resource  capacity  measure. 

Optimizing  Routines:  Solving  for  optimal  switch 
point  from  one  skill  to  the  next  related  skill. 

6.  Cost/Benefit  Structure  Determination:  Transfer 
considerations  etc. 
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Preliminary  Analysis 

Auto-Speech  Technology  Based  Training  Systems 


I*  Tasks— In  Context 

A.  Cues 

1.  Cues  are  primarily  visual:  e.g.  A  radar  screen 

(GCA, AIC)  or  the  actual  aircraft  (LSO). 

2.  Some  verbal  cues:  e.g.  Verbal  responses  from 

approaching  aircraft  (LSO)  or  pilot  (AIC). 

3.  Some  cues  internal:  e.g.  Temporal  cues — cer¬ 

tain  number  of  calls  per  unit  time. 

B.  Response  Requirements 

1.  Verbal  responses— All  three  require  this. 

2.  Motor  responses— Only  the  AIC  task  requires 

buttons,  foot  pedals,  etc.,  to  be  depressed 
during  operations. 

C.  Temporal  Requirements 

1.  Unpaced:  All  require  some  "unhurried1' 

responding,  e.g.  accepting  a  handoff  from 
another  controller  in  GCA-CTS. 

2.  Paced:  Responses  are  demanded  within  tem¬ 

poral  constraints  as  required  by  speed  of 
approaching  aircraft. 

II.  Analysis  of  Tasks 

A.  Visual  Componants 

1 .  Above  threshold 

2.  Divided  displays:  Scanning  responses  required 

(GCA) 

3.  Perceptual  Cues  Subtle:  Some  perceptual 

learning  required  (LSO) 

4.  Motion  (particularly  LSO) 


Ik 
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B.  Auditory  Cues 

1 •  Above  threshold 

2.  Background  Noise? 

C.  Temporal' Requirements 

1 •  Dual  task  situation? 

Encoding,  Search/Comparison,  and  Response 
Selection  is  simultaneous  with  Response 
Production  (verbal  and/or  motor)  leading 
to  capacity  limitation  considerations. 

2.  Encoding 

Uncertain  about  effective  cues? 

Some  cues  may  be  unitized  quite  early? 

e.g.  GCA  Target,  Intersect,  Trail  combo. 

3.  Search/Comparison 

a)  Visual  Search:  Simultaneous  Inputs  will 
cause  divided  attention  deficits  with  con¬ 
trolled  processing  early  in  training. 

b)  Memory  Search:  Categorical  structure  of 
memory  set? 

4«  Response  Selection:  Internal  schemata  unknown 
but  could  be  of  considerable  extent.  Related 
to  linguistic  structure  of  response. 


a 
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III.  Anecdotal  Observations 

A.  "Expert" 

1.  Notes  scanning  and  processing  while  verbalizing 

2.  Rapid  scanning  of  display 

3.  Unaware  of  many  functions  indicating  rapid 

automatic  processing 

4.  Learns  expectancies  (anticipates) 

B.  "Novice" 

1.  Pauses  in  verbalizations:  e.g.  "Turn. .right., 
heading.. one......./  /... five. .. five"  indi¬ 

cating  degradation  of  verbal  task  because  of 
extensive  controlled  processing. 
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2.  Intrusions  of  incorrect  terminology:  Response 
selection  errors  could  signify  several  pro- 
blems,  e.g.  interrupted  processing,  incorrect 
rule  structure/schemata? 

3#  Omissions:  Controlled  processing  not  completed 
in  time? 

Inappropriate  scanning  responses:  Students 
will  stare  at  small  portion  of  display. 
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TABLE  B1 .  ABBREVIATIONS  OF  THE  LIMITED  VOCABULARY 
TO  BE  USED  BY  THE  TRAINEE 


.  . 

- —  -  -  - 

w 

well 

S 

slightly 

C 

coming  G 

going 

A 

above 

0 

on 

U 

up  F 

further 

B 

below 

Gp 

Glidepath 

D 

down 

TABLE  B-2.  VALID  POSITION  ADVISORIES 


well  above  glidepath 

WAGp 

slightly  below  glidepath 

SBGp 

above  glidepath 

AGp 

below  glidepath 

BGp 

slightly  above  glidepath 

SAGp 

well  below  glidepath 

WBGp 

on  glidepath 

OGp 

TABLE  B-3.  VALID  TREND  STATEMENTS  BASED  ON 
OLD  AND  NEW  POSITIONS 


- •  -  - 

- -  - — - 

—  - _  .  - 

_  _  - • 

Old 

New 

Trend 

Old 

New 

Trend 

Position 

Position 

Statement 

Posit ion 

Position 

Statement 

WAGp 

WAGp 

None 

OGp 

SBGp 

GBGp 

WAGp 

AGp 

CD 

SBGp 

OGp 

CU 

AGp 

WAGp 

GFAGp 

SBGp 

BGp 

GFBGp 

AGp 

SAGp 

CD 

BGp 

SBGp 

CU 

SAGp 

AGp 

GFAGp 

BGp 

WBGp 

GFBGp 

SAGp 

OGp 

CD 

WBGp 

BGp 

CU 

OGp 

SAGp 

GAGp 

WBGp 

WBGp 

None 
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